News, Blogs, and Tips

Fun With Testing DateUtils.pas #7

Nick Hodges - Wed, 04/14/2010 - 08:03

Okay, so after that last post, and after all the fixes that I did, things have settled down a little bit.  I thought I’d take advantage of this interlude to tell you some interesting things about TDateTime in Delphi, because along the way here, I have discovered a thing or two along that way that was surprising.

The first thing that you might be interested in is that the calendar used by the Delphi TDateTime has a specific name:  The Proleptic Gregorian Calendar.  Calendars, of course, have been notoriously inaccurate over the years, and even ours isn’t entirely accurate in that we have to have leap years every so often (and not as often as one might believe…). We even have these “leap seconds” every once and a while, though the notion of being able to measure things down that precisely is kind of weird.  Starting with the Romans – Julius Caesar, actually – the Western world used the Julian calendar.  And that worked pretty well, actually.  The Julian calendar worked pretty well – it has 365 days and a leap year every four years – but it wasn’t entirely accurate, and (as you can read in the Wikipedia entry) politics got involved, and the calendar got out of whack pretty easily and pretty often.

So anyway, as you may have noticed, some of the tests that I have written include some pretty big time increments – like 1000 years worth of seconds and things like that.  And I also wanted to makes sure that the incrementing routines worked across the epoch of December 30, 1899.  So I had to be able to do some pretty serious calculations.  I found a pretty good website to do those calculations called  This site has a bunch of calculators for measuring between dates and time and for calculating a date based on distance from another date.  So I used it to figure out what the date was if you decremented two hundred years worth of seconds (that’s 6,307,200,000   seconds for you math geeks….) from, say, Noon on June 30, 1945.  (It’s not exactly noon on  June 30, 1745 due to leap days.)  Well, I would calculate it, and then write the tests, but they would fail because my expected result was always eleven days different than the test result.  Eleven days – how weird, huh?

Well, here’s the deal.  Somewhere along the way, the really smart guys who figure out this kind of thing came up with a new calendar – the Gregorian calendar.  It’s different from the Julian calendar, and starting in the 16th century, the world gradually converted over to use the Gregorian Calendar instead of the Julian calendar (A good chunk of Europe started in 1582, and the last folks to make the switch were the Russians who didn’t change until 1918).  But to do that, you usually had to skip about 10 or 11 days.  Great Britain and all of its possessions (including the colonies that would become the United States) made the switch in 1752.  Therefore, in the English world, the day following September 2, 1752 was September 14, 1752.  There was no September 3 – 13, 1752.  Just didn’t exist.  Once I discovered that, it explained the missing eleven days.

But what does this mean for our trusty TDateTime?  For a minute there I was afraid that I was going to have to do all these special calculations to account for this unusual anomaly, but then I came to my senses and realized:  That can’t be right.  And I was right.  Instead, Delphi uses, as I mentioned above, the Proleptic Gregorian Calendar – that is, it assumes that the Gregorian calendar is in force all the way  back to January 1, 0001.  So for TDateTime, there is a September 4, 1752 (Noon on that day is the value: -53807.5) and every single date “like normal” all the way down to Year 1.  This makes sense, because trying to devise a calendaring system that keeps track of all the vagaries of the Julian calendar system would be basically impossible.  Instead, Delphi uses a system that “makes sense” for a computer.  A number of other languages and tools use the Proleptic Gregorian Calendar, including MySQL and PHP.

That was probably more than you wanted to know about TDateTime, but it’s all stuff that you have to know to write a complete test suite for DateUtils.pas. So far, that summarizes the issues that I’ve run across in testing the unit. I have a ways to go to have a complete test suite, but if I run across more issues, I’ll post on them.

The next post I do will be about a testing scheme that one of our developers, Alexander Ciobanu, devised to make writing tests for testing date functions a little easier.

Categories: News, Blogs, and Tips

Fun With Testing DateUtils.pas #6

Nick Hodges - Tue, 04/13/2010 - 15:55

Okay, so when we last left off, IncMillisecond was still failing in certain circumstances.  Let’s take a look at that.  Note, too, that I have this crazy notion that if you have a function called IncMillisecond, then it should be able to, you know, increment a millisecond.

Here is the IncMilliseconds that you very likely have on your computer:

function IncMilliSecond(const AValue: TDateTime; const ANumberOfMilliSeconds: Int64): TDateTime; begin if AValue > 0 then Result := ((AValue * MSecsPerDay) + ANumberOfMilliSeconds) / MSecsPerDay else Result := ((AValue * MSecsPerDay) - ANumberOfMilliSeconds) / MSecsPerDay; end;

Now that probably works just fine for you — as long as you don’t have a date that has a value less than the epoch. Below the epoch, and particularly in that magic "48 Hours" area right around the epoch itself, things go horribly awry. As we saw last time, this test will fail:

TestDate := 0.0; TestResult := IncMillisecond(TestDate, -1); Expected := EncodeDateTime(1899, 12, 29, 23, 59, 59, 999); CheckTrue(SameDateTime(Expected, TestResult), 'IncMillisecond failed to subtract 1ms across the epoch');

It fails because of a number of reasons actually. The first is precision. The current implementation of IncMillisecond does division using a very small number in the denominator.  In the case of this test the numerator is a really big number multiplied by a really small number.  All of this cries out “precision error!”. (You should thank me – I almost used the <blink> tag there.  Phew!)  And that is basically what happens.  IncMillisecond isn’t precise enough to “see” the difference.

Plus, if you do things around the value of zero, it gets really weird.  For instance, check out the output of this console application:

program IncMillisecondTest; {$APPTYPE CONSOLE} uses SysUtils, DateUtils; var TestDate: TDateTime; TestResult: TDateTime; DateStr: string; begin TestDate := 0.0; TestResult := IncMilliSecond(TestDate, 1001); DateStr := FormatDateTime('dd mmmm, yyyy hh:mm:ss:zzz', TestResult); WriteLn(DateStr); TestResult := IncMilliSecond(TestDate, -1001); DateStr := FormatDateTime('dd mmmm, yyyy hh:mm:ss:zzz', TestResult); WriteLn(DateStr); ReadLn; end.

I think it is safe to say that something is amiss.

So finally, it is time to rework IncMillisecond, because this pesky little routine is actually at the heart of a bunch of issues with DateUtils.pas. As it will turn out, if you call any of the IncXXXX routines, it all ends up as a call to IncMilliseconds, so this needs to be right.

Okay, so I started out writing this really cool implementation that checked for before and after the epoch, and divided large increments into years and months and days to make sure that their was no loss of precision.  I spent a lot of time on it, and had  whole bunch of tests written and passing with it.   But then it suddenly occurs to me that the trusty TTimeStamp data type and its accompanying conversion routines can once again come to the rescue:

function IncMilliSecond(const AValue: TDateTime; const ANumberOfMilliSeconds: Int64 = 1): TDateTime; var TS: TTimeStamp; TempTime: Comp; begin TS := DateTimeToTimeStamp(AValue); TempTime := TimeStampToMSecs(TS); TempTime := TempTime + ANumberOfMilliSeconds; TS := MSecsToTimeStamp(TempTime); Result := TimeStampToDateTime(TS); end;

And here is the cool thing:  I was able to change from my sweet but overly complicated version to the new version above without worrying too much about it, because when I made the switch – all of the tests that I had written for my original version still passed.  This was so cool – I could make the change with confidence because of the large set of tests that I had that exercised all aspects on IncMillisecond.

Anywhow….  Again, the TTimeStamp type is precise, and easy. No need to do direct arithmetic on the TDateTime itself. Instead, we can deal with integers and get the exact answer every time no matter how many milliseconds you pass in. You can pass in 5000 years worth of milliseconds, and all will be well. For instance, this test passes just fine.

TestDate := EncodeDate(2010, 4, 8); MSecsToAdd := Int64(5000) * DaysPerYear[False] * HoursPerDay * MinsPerHour * SecsPerMin * MSecsPerSec; // 1.5768E14 or 157680000000000 TestResult := IncMilliSecond(TestDate, MSecsToAdd); Expected := EncodeDate(7010, 4, 8); ExtraLeapDays := LeapDaysBetweenDates(TestDate, Expected); Expected := IncDay(Expected, -ExtraLeapDays); CheckTrue(SameDate(Expected, TestResult), 'IncMillisecond failed to add 5000 years worth of milliseconds.');

And for you curious folks, here the implementation for the helper function LeapDaysBetweenDates:

function TDateUtilsTests.LeapDaysBetweenDates(aStartDate, aEndDate: TDateTime): Word; var TempYear: Integer; begin if aStartDate > aEndDate then raise Exception.Create('StartDate must be before EndDate.'); Result := 0; for TempYear := YearOf(aStartDate) to YearOf(aEndDate) do begin if IsLeapYear(TempYear) then Inc(Result); end; if IsInLeapYear(aStartDate) and (aStartDate > EncodeDate(YearOf(aStartDate), 2, 29)) then Dec(Result); if IsInLeapYear(aEndDate) and (aEndDate < EncodeDate(YearOf(aEndDate), 2, 29)) then Dec(Result); end;

From there, the rest of the IncXXXXX routines are simple –- they merely multiply by the next “level up” of time intervals, and call the previous one.  I’ve marked them all inline so that it all happens in one need function call.  Thus, we have:

function IncHour(const AValue: TDateTime; const ANumberOfHours: Int64 = 1): TDateTime; begin Result := IncMinute(AValue, ANumberOfHours * MinsPerHour); end; function IncMinute(const AValue: TDateTime; const ANumberOfMinutes: Int64 = 1): TDateTime; begin Result := IncSecond(AValue, ANumberOfMinutes * MinsPerHour); end; function IncSecond(const AValue: TDateTime; const ANumberOfSeconds: Int64 = 1): TDateTime; begin Result := IncMilliSecond(Avalue, ANumberOfSeconds * MSecsPerSec); end;

One thing to note: DateUtils.pas will only handle years from 1 to 9999. TDateTime won’t handle any date less than midnight on January 1, 0001 nor a date larger than December 31, 9999. So if you are using Delphi to track specific dates in dates before that (or if you plan on doing some time travel into the far future) you’ll have to use some other data type to keep track of dates.

Now, once you’ve done the above, it is tempting to say “Hey, for IncDay, I’ll just add the days to the value passed in.  I mean, that’s all you are really doing.  Well guess what!  You can’t do that!  If you have this for your IncDay:

function IncDay(const AValue: TDateTime; const ANumberOfDays: Integer = 1): TDateTime; begin Result := AValue + ANumberOfDays; end;

Then this test will not pass because of the strange “48 hour” deal we talked about last post:

TestDate := EncodeDateTime(1899, 12, 30, 1, 43, 28, 400); TestResult := IncDay(TestDate, -1); Expected := EncodeDateTime(1899, 12, 29, 1, 43, 28, 400); CheckTrue(SameDate(Expected, TestResult), 'IncDay failed to decrement one day from the epoch');

Instead, you have to send it all the way back to milliseconds via IncHour, IncMinute, and IncSecond:

function IncDay(const AValue: TDateTime; const ANumberOfDays: Integer = 1): TDateTime; begin Result := IncHour(AValue, ANumberOfDays * HoursPerDay); end;

Once you put those changes in, well, things get a lot greener.  I have now written a very thorough set of unit test  for testing all of the IncXXXX routines, adding and subtracting dates for both before and after the epoch.  I also test very carefully incrementing and decrementing across the epoch and inside that crazy little 48 hour spot.  They are all passing.

I’ll create a unit with these new fixes in it that you can use if you want.  I’ll also publish the unit that includes these tests that I’ve written.  (When you look at it, be nice.  It’s not very pretty, but it gets the job done.)  As I continue through, I’ll update that file with any other fixes and changes that get made.

Categories: News, Blogs, and Tips

Fun With Testing DateUtils.pas #5

Nick Hodges - Wed, 04/07/2010 - 10:02

Okay, so when I left you hanging in the last post, I promised I’d explain what was up with IncMillisecond.  But before I do that, I have to explain a bunch of stuff about TDateTime. And as it turns out, we’ll have to take a detour, and we won’t exactly get to IncMillisecond this time around. 

Most of you probably know how TDateTime works.  TDateTime is a Double that keeps track of minutes in the “front” of the decimal and seconds in the fraction, or the “back”.  The key thing to know is the value of the “epoch” that I mentioned previously.  For TDateTime, the epoch is 0.0, which corresponds to exactly 00:00:00.000 (midnight) on December 30, 1899. (For can read up on all the gory details about why it is December 30, 1899, and not December 31, 1899

What this means is that a date time of 2.0 is January 1, 1900 at midnight.  2.5 would be noon on January 1, 1900.  1000.25 would be one thousand days and six hours past December 30, 1899, or September 26, 1903 at 6:00:00 AM.  It also means that –1 is December 29, 1899.  and –1000.25 is Sunday, April 4, 1898 at 6:00:00 AM. 

Now, that last one was a bit tricky if you look carefully at it.  The days part was negative (-1000) but the hours part was not.  Remember, the left part of the double is the number of days before the epoch, but the decimal part – the part to the right, if you will – is always a positive value starting at midnight of the day in question.   I emphasized that last part pretty strongly because once a date goes negative, a counter intuitive thing happens.  The negative part only really applies to the left portion of the value.  The decimal value represents a positive value from midnight.  So to do the last calculation above, I actually had to subtract 999 days and 18 hours to get the right answer.  And there in lies the heart of the problem that we have run into with incrementing milliseconds (and seconds and minutes and hours, as it turns out) for days before the epoch. 

Here’s another way to think about it:  what is the date time value for –0.5?  Well, the correct answer is noon on 29 December 1899.  But look at the left part of the value – it is still zero, which is, of course, 30 December 1899!  And what if you make the call Frac(-0.5) to that value?  You get – ready for it? — -0.5!  And I just got done telling you that you can’t have a “negative” time value.  Time values always are positive values from midnight.  And herein lies our problem. 

Another interesting note:  In the particular world of TDateTime, 0 has an unusual “feature”.  When viewed as the “left” side of a TDateTime, it actually represents a span of time just a hair less than 48 hours.  According to the pure mathematical formula for managing dates and times in Delphi, December 30, 1899 actually has 48 hours.  That is, it stretches from –0.999… to 0.999…. in time.  This is weird, huh?  Never really thought about that, did you?  Well, the whole Date/Time system has to account for this little anomaly. 

So, we have two related issues here:  Time values for negative TDateTime values are really positive, and this weird 48 hour day thing right at the epoch.  Well, frankly I didn’t think about or know about either one when I started out writing my unit tests (until they revealed this issue to me.  Unit testing rocks…) and I am very sad to say that the original author of DateUtils.pas didn’t either.  Both of these errors manifest themselves when calculating times at and before the epoch.  That’s the bad part.    And I know all of this because of unit testing.  That’s the good part. 

But wait, there is more.  As it turns out, all of the time calculations in DateUtils.pas are based on floating point values.  Very, very small floating point values, in fact.  For instance, take a look at the current implementation of IncMillisecond:

function IncMilliSecond(const AValue: TDateTime; const ANumberOfMilliSeconds: Int64): TDateTime; begin if AValue > 0 then Result := ((AValue * MSecsPerDay) + ANumberOfMilliSeconds) / MSecsPerDay else Result := ((AValue * MSecsPerDay) - ANumberOfMilliSeconds) / MSecsPerDay; end;

The value for MSecsPerDay is pretty large — 86,400,000 – and when you start dividing small numbers by really big numbers you get even smaller numbers –numbers so small that they lose precision.  Now, you can see that our developer at least recognized that something  was going a little goofy with the dates before zero, but the current implementation has the error we are currently looking at.  Alas.

Or even better, go to SysUtils.pas and take a look at TryEncodeTime, which really does some arithmetic fraught with the possibilities for errors and inaccuracies:

function TryEncodeTime(Hour, Min, Sec, MSec: Word; out Time: TDateTime): Boolean; begin Result := False; if (Hour < HoursPerDay) and (Min < MinsPerHour) and (Sec < SecsPerMin) and (MSec < MSecsPerSec) then begin Time := (Hour * (MinsPerHour * SecsPerMin * MSecsPerSec) + Min * (SecsPerMin * MSecsPerSec) + Sec * MSecsPerSec + MSec) / MSecsPerDay; Result := True; end; end;

That will create some seriously small values, won’t it, given data near midnight on either side?  I’ve subsequently reworked this routine to be more precise.  (I’ll post all this new code for your real soon now.)

Okay, so where to turn in all of this?  The first thing I did was to rewrite IncMilliseconds.  But as you’ll see, even this was really, really tricky and fraught with peril as well.

Okay, so I thought – I’m doing all this test driven development; what I need to do right now is to write some test cases that I know should pass before I even start.  First, I thought that if you have a function called IncMillisecond, then it ought to at least have enough accuracy and precision to at the very least create a different date/time combination, right?

TestDate := 0.0; TestResult := IncMillisecond(TestDate); CheckFalse(SameDateTime(TestDate, TestResult), 'IncMilliseocnd failed to change the given date');

And of course, this fails.  Good – I expected it to. But after a few hours of writing code, and wondering why it keeps failing, I suddenly realize that SameDateTime is the problem here!  Argh!

And then it hits me – Uh oh.  I’ve started pulling on a thread, and if I keep pulling on it, it is going to keep unraveling and unraveling….  And that is exactly what happened.

Checkout your SameDateTime:

function SameDateTime(const A, B: TDateTime): Boolean; begin Result := Abs(A - B) < OneMillisecond; end;

Now, that looks all well and good. Take the absolute value of the difference, and as long as it is less than 1ms, then the times are effectively the same. OneMillisecond is defined as: OneMillisecond = 1 / MSecsPerDay, or 1.15740741 × 10-8. And in the world of computers, that is a pretty small number. So small, in fact, that it is pretty easy to have small values not register. In our simple test here, the A value is 0, and the B value -1.1574074074e-08. And guess what, that difference is not quite enough to get SameDateTime to return False. It returns True instead.

So, let’s follow this loose thread a bit more, and then we’ll quit for today. We need a SameDateTime function (and, as it turns out, a SameTime function) that returns a correct answer for dates that actually are OneMillisecond apart. We need something that gives answers based on real number so of milliseconds.  And SysUtils.pas has the answer:  TTimeStamp

TTimeStamp is declared as follows:

{ Date and time record } TTimeStamp = record Time: Integer; { Number of milliseconds since midnight } Date: Integer; { One plus number of days since 1/1/0001 } end;

Now, that is more like it — integers and not these fuzzy floating point numbers! The accompanying DateTimeToTimeStamp function is exactly what we need. Now, we can write a very precise SameDateTime and SameDate functions:

function SameDateTime(const A, B: TDateTime): Boolean; var TSA, TSB: TTimeStamp; begin TSA := DateTimeToTimeStamp(A); TSB := DateTimeToTimeStamp(B); Result := (TSA.Date = TSB.Date) and (TSA.Time = TSB.Time); end; function SameTime(const A, B: TDateTime): Boolean; begin Result := (DateTimeToTimeStamp(A).Time = DateTimeToTimeStamp(B).Time); end;

Those two new implementations will, in fact, return correct results for two dates one millisecond apart.  And let’s just say that TTimeStamp is going to be making more appearances in the new, updated DateUtils.pas in the future.

Okay, so our original, simple test above passes now. But guess what: this second one still doesn’t:

TestDate := 0.0; TestResult := IncMillisecond(TestDate, -1); CheckFalse(SameDateTime(TestDate, TestResult), 'IncMilliseocnd failed to change the given date'); Expected := EncodeDateTime(1899, 12, 29, 23, 59, 59, 999); CheckTrue(SameDateTime(Expected, TestResult), 'IncMillisecond failed to subtract 1ms across the epoch');

So next time, we’ll get cracking on that.

Categories: News, Blogs, and Tips

Fun With Testing DateUtils.pas #4

Nick Hodges - Thu, 04/01/2010 - 14:39

First, an admin note:  I’ve adjusted the color of strings in my code.   I was optimizing the colors for reading on my blog proper as opposed to the main site (hadn’t even thought of it, actually, sorry.), and someone pointed out that the colors weren’t working on the main site at all.  Hope that this post is better.  I changed the last post from Yellow to Lime.  If you have a better color suggestion, please let me know.  I’ve also endeavored to wrap those long code lines. The code won’t compile as shown, but I trust that you guys can figure it out……

Okay back to the topic at hand.

So things are rolling along.  I’ve been writing tons of tests, they are all passing, things are going well, and it’s been fun.

But if you have any flair for the dramatic, you can see where this is going….

So there I was rolling along, writing tests for WeeksInAYear (bet you didn’t know that according to ISO 8601, some years have 53 weeks in them, did you.  1981 has 53 weeks, for example) Today, Yesterday – you know, normal stuff.  I’m checking edge conditions, standard conditions, all kinds of years, every year.  You know, really exercising things.  All was rolling along smoothly.

For instance, here are the tests for Yesterday.  Not too hard to test, as there is really only one thing you can do:

procedure TDateUtilsTests.Test_Yesterday; var TestResult: TDateTime; Expected : TDateTime; begin TestResult := Yesterday; Expected := IncDay(DateOf(Now), -1); CheckEquals(TestResult, Expected, 'The Yesterday function failed to return the correct value.'); TestResult := Yesterday; Expected := DateOf(Now); CheckFalse(SameDate(TestResult, Expected), 'The Yesterday function thinks Yesterday is Today, and means that Einstein was totally wrong.'); end;

Just a couple of tests that you can do – or at least what I can think of.  (Anyone have any other ideas?)  The fun part is that these tests will fail if IncDay and DateOf fail to perform as advertised, we get triple the testing!  Sweet!

Things were going along swimmingly, and then all of a sudden, out of left field, all this unit testing stuff suddenly proved to be as valuable as everyone says it is.

Here’s how it happened: I was going along, writing tests, and I wrote this one:

procedure TDateUtilsTests.Test_EndOfTheDay; var TestDate : TDateTime; TestResult: TDateTime; i : Integer; Expected : TDateTime; begin for i := 1 to 500 do begin TestDate := CreateRandomDate(False, 100, 2500); TestResult := EndOfTheDay(TestDate); // First, don't change the date CheckEquals(DayOf(TestDate), DayOf(TestResult), Format('EndOfTheDay changed the day for test date: %s (Result was: %s)', [DateTimeToStr(TestDate), DateTimeToStr(TestResult)])); // Next, is it really midnight? Expected := DateOf(TestDate); Expected := IncMillisecond(Expected, -1); Expected := IncDay(Expected); CheckTrue(SameDateTime(TestResult, Expected), Format('EndOfTheDay didn''t return midnight for test date: %s (Result was: %s, Expected was: %s)', [DateTimeToStr(DateOf(TestDate)), DateTimeToStr(TestResult), DateTimeToStr(Expected)])); end; end;


Pretty simple and straightforward.  But — BOOM – this thing fails. Badly.  If you run this test on your computer, the second check, the call to CheckTrue, will pretty quickly fail and you’ll get a message something like:

Test_StartEndOfTheDay: ETestFailure at  $0051FF06 EndOfTheDay didn’t return midnight for test date: 5/12/0366 (Result was: 5/12/0366 11:59:59 PM, Expected was: 5/14/0366 11:59:59 PM), expected: <True> but was: <False>

Since the test is creating random dates, you’ll never get the exact same error, but pretty soon I figured out that it only failed for dates before the epoch – that is, for dates that have a negative value and are thus earlier than 30 December 1899. 

Naturally, I was left scratching my head.  The first inclination is that the test is somehow not correct. But I stared at it for a good long while and came to the conclusion that the test wasn’t the problem. 

The first check is fine – the call to EndOfTheDay doesn’t actually change the date as it shouldn’t.  But the second test is where the trouble started. 

EndOfTheDay is a pretty simple function;  it returns the very last millisecond of the date for the date/time combination passed to it – that is, 11:59.999pm for the day in question. It is implemented like so:

// From DateUtils.pas function EndOfTheDay(const AValue: TDateTime): TDateTime; begin Result := RecodeTime(AValue, 23, 59, 59, 999); end;

So the natural thing is to actually check to see if the result is indeed that value.  So, I did the natural thing:  I set the expected date to midnight on the date of the value to be tested, decremented one millisecond, and since that changed the date back one day, I moved it forward again with IncDay.  Then I checked to see if they were indeed the same date/time combination.  Well, guess what.  They weren’t. 

I originally had a single line of code combining the three that set the value for Expected.  A quick look at the debugger told me that the Expected result wasn’t getting properly calculated.  Breaking it down quickly pointed to a strange phenomenon:  for dates before the epoch, the IncMillisecond call was actually moving the date portion forward  by two days if the date was before the epoch.  (Mysteriously, dates after epoch all worked fine.  Weird.)  That, of course, is a big bad bug. 

And this is the part where using the library itself to test other parts of the library is helpful.  Because I used IncMillisecond in my test for EndOfTheDay, I found a bug in IncMillisecond. If I hadn’t done so, the problem might have been left lurking for a while longer.  Or maybe it never would have revealed itself, depending on how diligent my testing of it ended up once I actually got there. 

Luckily, it would appear that not too many of you are manipulating milliseconds for dates before the epoch, because there hasn’t been a big hue and cry about this problem. There have been some QC reports about it, though.  But clearly something is dreadfully wrong here. 

In the next post, we’ll take a look at just what that is.

Categories: News, Blogs, and Tips

Random Thoughts on the Passing Scene #153

Nick Hodges - Wed, 03/31/2010 - 17:03
  • Andreano has a new blog, or at least it is new to me – just found it today.  He has two items there that caught my eye:
  • The inestimable Mike Rozlog is on a hot streak, and he continues to give cool and interesting Webinars.  His latest is “Mastering Database Application Development with Delphi”.  Once again, he’ll be giving the webinar at three times during the day of 14 April 2010, so no matter where you are in the world, you should be able to attend one of them. 
  • These marketing people are busy.  What to know about RAD In Action regarding building database applications?  There’s a web page for that
  • Julian Bucknall, the CTODX (Chief Technology Officer of Developer Express) has an update for their VCL customers.
  • My mentioning of our move caused a bit of a stir in the comment section.   A couple of more thoughts on it.  I can only speak for myself, but so far it is working out pretty well.  I like the new space.  I like my cube.  I like that everyone is fairly close together but not too close.  Our previous space was waaaaay  to big for us, and you could go weeks without seeing someone from Sales or Support.  Now, we are are all in one space, and it feels more like we are one team, which of course we are.  I like that this new place is a good fit.  I like that this place is significantly more appropriate and significantly less expensive that our previous space.  I like that this place is a new start. I like that we have a gigabit network.  I like that we have projectors hanging from the ceilings in the conference rooms.  I like that it is closer to the shopping mall across the street. But most of all, I like that it represents a significant investment in and commitment to our team. So for me, this is a big win.
Categories: News, Blogs, and Tips

Fun With Testing DateUtils.pas #3

Nick Hodges - Tue, 03/30/2010 - 19:20

Okay, things have settled down again, and it is time to get back to my adventure in TDateTime and DateUtils.pas.

When we last left off, I had started at the top of DateUtils, and just started working my way down.  I had written some tests for DateOf and TimeOf, and tried to write tests that pretty thoroughly exercised those functions.  I tried to hit the edges and boundaries, and to test all the different permutations and combinations of a date only, a time only, and both together. 

From there, I worked my way down the list, writing tests for IsLeapYear, IsPM, etc. 

One thing I did was to add IsAM to DateUtils.pas and simply implemented it as:

function IsAM(const AValue: TDateTime): Boolean; begin Result := not IsPM(AValue); end;

Now, that is really simple.  Shoot, you don’t really need to write tests for that, right?  I mean, I wrote a whole suite of tests for IsPM, and so how could IsAM go wrong? Well, any number of ways – but the main one is that some day in the future, someone might come along and try to get cute or super-smart or something and change the implementation.  So I went ahead and wrote a whole bunch of tests for IsAM anyway.  Now, if something changes, or if someone changes something, the tests should be able to recognize that. 

Philosophical Note: As I’m doing this, I’m seeing more clearly than ever that writing tests is all about confidence moving forward.  Once you have taken the effort to write thorough, complete suites of unit tests, you can move forward with confidence.  You can make changes and fixes while feeling confident that if your change has unintended consequences, you’ll likely know about it. If you do find a bug, you write a test that “reveals” it, fix the bug so the test passes, and then you can move forward confident that you’ll know right away if that bug comes back to haunt you.  Confidence is a really good thing when it comes to writing code.

So, for instance, let’s look at the tests for IsInLeapYear.  Leap years are a bit funky.  Some years that you think are leap years are not – Quick:  Was 1600 a leap year?  What about 1900?  Wikipedia actually has a good page on leap years.  (Did you know that leap years are also called “intercalary years”? I sure didn’t.)  The actual calculation of a leap year is a bit more complicated that “Is it divisible by 4?”. 

function IsLeapYear(Year: Word): Boolean; begin Result := (Year mod 4 = 0) and ((Year mod 100 <> 0) or (Year mod 400 = 0)); end;

Examine the code, you can see that the answer to the questions above are Yes and No.  (As a side note, our QA Manager is a “Leapling”, born on February 29th.  He’s really only 12 years old.)

So, how do you test something called IsInLeapYear?  The declaration is actually quite simple:

function IsInLeapYear(const AValue: TDateTime): Boolean; begin Result := IsLeapYear(YearOf(AValue)); end;

But just because it is simple doesn’t mean that you shouldn’t thoroughly test it!  So I wrote a whole bunch of tests. First, I checked that random dates in years I know are leap years were properly identified as being in a leap year:

TestDate := EncodeDate(1960, 2, 29); TestResult := IsInLeapYear(TestDate); CheckTrue(TestResult, Format('%s is in a leap year, but IsInLeapYear says that it isn''t. Test #1', [DateToStr(TestDate)])); TestDate := EncodeDate(2000, 7, 31); TestResult := IsInLeapYear(TestDate); CheckTrue(TestResult, Format('%s is in a leap year, but IsInLeapYear says that it isn''t. Test #2', [DateToStr(TestDate)])); TestDate := EncodeDate(1600, 7, 31); TestResult := IsInLeapYear(TestDate); CheckTrue(TestResult, Format('%s is in a leap year, but IsInLeapYear says that it isn''t. Test #4', [DateToStr(TestDate)])); TestDate := EncodeDate(1972, 4, 5); TestResult := IsInLeapYear(TestDate); CheckTrue(TestResult, Format('%s is in a leap year, but IsInLeapYear says that it isn''t. Test #5', [DateToStr(TestDate)])); TestDate := EncodeDate(1888, 2, 29); TestResult := IsInLeapYear(TestDate); CheckTrue(TestResult, Format('%s is in a leap year, but IsInLeapYear says that it isn''t. Test #7', [DateToStr(TestDate)])); TestDate := EncodeDate(2400, 2, 29); TestResult := IsInLeapYear(TestDate); CheckTrue(TestResult, Format('%s is in a leap year, but IsInLeapYear says that it isn''t. Test #8', [DateToStr(TestDate)]));

Note that I checked "normal" dates, but also dates in the far future (including the tricky 2400) as well as dates before the epoch (which is December 30, 1899, or a datetime value of 0.0). I’ll talk a little more about the epoch in a future post because the epoch is really, really important to TDateTime. It is also really, really troublesome. 

Another thing to note is that this code uses (and thus tests) EncodeDate. And IsInLeapYear itself will exercise YearOf and IsLeapYear indirectly.  If a test in IsInLeapYear fails indirectly because of one of these, you’ll be able to figure that out pretty quickly, write tests specifically to reveal those problems, fix the problems, and then move forward with confidence that you’ve resolved the issues.

Anyway, I also wrote some negative test cases, checking to see that it returned False for dates that most definitely were not in leap years.   I also wrote tests for dates in years that many folks might thing are leap years but are in fact not leap years:

// Years that end in 00 are /not/ leap years, unless divisible by 400 TestDate := EncodeDate(1700, 2, 28); TestResult := IsInLeapYear(TestDate); CheckFalse(TestResult, Format('%s is in a leap year, but IsInLeapYear says that it isn''t. Test #6', [DateToStr(TestDate)])); TestDate := EncodeDate(1900, 2, 28); TestResult := IsInLeapYear(TestDate); CheckFalse(TestResult, Format('%s is in a leap year, but IsInLeapYear says that it isn''t. Test #7', [DateToStr(TestDate)]));

Now that might seem like overkill for a simple function like IsInLeapYear, but I don’t think so. I am now really confident that, since we will be running these tests almost continuously on our Hudson server, no one can mess or alter or change or otherwise break the way leap years are calculated without us knowing about it immediately. And that’s sort of the whole point, right?

Categories: News, Blogs, and Tips

Random Thoughts on the Passing Scene #152

Nick Hodges - Tue, 03/23/2010 - 14:08
  • As you may know, we are moving to a new office very nearby where we are now.  We got to visit the new digs today.  Friday is moving day, and we’ll be in the new place for a great new start on Monday morning.  I’m excited – I think that the move will be a great fresh start for us in a new place.  We’ll be moving from offices to cubes, so that will be a bit of a culture shift and an adjustment, but it should be great.  Pursuant to the move, of course, we have been and will be migrating servers to our new location.  So if things aren’t working 100% correctly, give it a bit and try again.  If things are persistently not working, then let us know.  Our IT team is working very hard to make sure that the migration goes smoothly, but there will inevitably be hiccups along the way. Your patience and understanding are appreciated.
  • From time to time, people ask how to make a deep copy of an existing instance of a class.  Well, using the new, super cool RTTI, Alex is on the case.
  • Our Haiti Auction got written up in the San Jose Mercury News.  Nice!
  • New Delphi site offering specials on Delphi components:
  • One of the best parts about Delphi is the awesome community, and one of the best parts of the community is the JEDI team.  These guys are awesome, and provide and an incredible amount of value to all of us.  They had some newsgroups at that have apparently stopped working. As a result, they have a new server at where you can point your NNTP newsreader on port 119. 
Categories: News, Blogs, and Tips

Random Thoughts on the Passing Scene #151

Nick Hodges - Thu, 03/18/2010 - 16:22
Categories: News, Blogs, and Tips

Random Thoughts on the Passing Scene #150

Nick Hodges - Wed, 03/17/2010 - 08:59
  • David I is doing another webinar:  RAD/AJAX-based Web Application Development using RAD Studio 2010.  He is giving it three different times on Thursday, March 18, 2010, so no matter where you are in the world, you should be able to tune in.
  • Anders Ohlsson has wrapped up the Haiti auction, and when all is said and done, you all will have helped to contribute over US$26,000 to Haitian relief.  That’s amazing.  A huge thank you to all that bid and bought items, and to those that help spread the word.  You all were very generous and frequently paid quite a bit more than fair market value.  ;-)  Thanks, also, to folks like Anders Hejlsberg, Danny Thorpe, Chuck Jazdzewski, Zack Urlocker, Gary Whizin and others that made the effort to sign some of the items and increase their value.  A huge thank you to Anders who did the lion’s share of the work.  All in all, it was a great thing you all did.   Very cool.
  • Just two items today, as I wanted to get the David I seminar posted…..
Categories: News, Blogs, and Tips

Fun With Testing DateUtils.pas #2

Nick Hodges - Tue, 03/16/2010 - 10:26

First of all, thanks for the help improving the CreateRandomDate function.  I confess that I didn’t spend enough time thinking on it, and I’ll also confess that you guys are way smarter than I am.  ;-)  I’ll post the “updated” version for your perusal in a separate post. 

In addition, you all were right – that code formatting sucks.   The plug-in I got for Live Writer was not very configurable.  I am now using John Kasters “YAPP” tool, and it all looks a lot better. 

Okay, back to the show…..

So to get going with unit testing DateUtils.pas, I naturally simply “plugged in” to our existing RTL unit test framework.  We have an existing RTL set of unit tests for running DUnit tests on the RTL.  I simply added the unit UnitTest.DateUtils.pas to the project, created the new class:

TDateUtilsTests = class(TTestCase) end;

and I was in business.  From there, it’s merely a matter of declaring published methods that run the DUnit tests.

So, I started right at the top with DateOf and TimeOf. So, what to test? Well, the most obvious thing: Does DateOf actually return the date portion of a given TDateTime?  Well, lets create a TDateTime with a random time, then, lets create a TDate with the same date but no time at all, and see if DateOf can do it’s magic?

procedure TDateUtilsTests.Test_DateOf; var TestDate: TDateTime; Expected: TDateTime; Result : TDateTime; begin TestDate := EncodeDateTime(1945, 12, 1, 1, 46, 13, 112); Expected := EncodeDate(1945, 12, 1); Result := DateOf(TestDate); CheckTrue(SameDate(Result, Expected), 'Test date and Expected date were not the same.' + ' DateOf function failed. Test #1'); end;

So this is a pretty straightforward test – you create two dates, and see if they are the same after the call to DateOf.  Simple.

What if you, say, increment the time part by one millisecond.  Come on, that can’t hurt anything right?  Better make sure:

// This test will fail if it gets run at 23:59.999 at night. // I'm willing to gamble that this will never happen. TestDate := Now; Expected := IncMillisecond(TestDate); Result := DateOf(TestDate); CheckTrue(SameDate(Result, Expected), 'Test date and Expected date were not the same.' + ' DateOf function failed. Test #2');

Okay, those are some “positive test cases”.  (I have a bunch more different ones along these lines….)  What about testing the negative case?  That is, test where we know that the two dates should be different after the call, and we test to make sure that they are, indeed different.

TestDate := Now; Expected := DateOf(IncDay(TestDate)); Result := DateOf(TestDate); CheckFalse(SameDate(Result, Expected), 'Test date and Expected date should have' + ' been different but they weren''t. Test #2');

I have a similar set of tests for TimeOf.  These are pretty basic, but that is where you start, right?  With the basics?  From there, I wrote tests that change only the milliseconds, the seconds, the minutes and the hours.  All should never allow the DateOf function to return anything other than the date.  For TimeOf, I do the same – change the year, month, and date and make sure the time is the same.  Then I purposefully change the time and make sure that the function actually does change the time. 

Now, some of you are going to chastise me for using other DateUtils.pas functions to write tests.  Two schools of thought on that.  One says that you should never rely on anything outside of the actually call being tested.  Another says to use those library functions because they’ll get tested all the more when used in other tests. I’m going to be following the latter philosophy, and as well see in a later post, this way of doing things actually will reveal a pretty significant bug in a routine that was used to test another routine.

Categories: News, Blogs, and Tips