Last mod time from stat() can be wrong on Windows NT/2000/XP

[p5pRT]

Migrated from rt.perl.org#18513 (status was 'stalled')

Searchable as RT18513$

[p5pRT]

From @steve-m-hay

This is a bug report for perl from Steve.Hay@​uk.radan.com,
generated with the help of perlbug 1.34 running under perl v5.8.0.

When running on Windows NT/2000/XP Perl's built-in stat() function can
return the wrong last modification time of a file.

The value returned should be expressed as seconds since the Epoch, and
should therefore be independent of the system's time zone and daylight
savings time (DST) settings.

However, for a file created during a period when DST is in effect the
value actually returned when the system is no longer in a DST period
depends on whether or not the system is configured to automatically
make adjustments for DST.

Example​:

  Running on Windows XP Professional, Service Pack 1
  "Regional and Language Options" set to English (United Kingdom)

  Open the "Date and Time Properties" applet from Control Panel

  Go to the "Time Zone" tab

  Select "(GMT) Greenwich Mean Time​: Dublin, Edinburgh, Lisbon, London"
  and select the option to "Automatically adjust clock for daylight
  saving changes"

  Go to the "Date & Time" tab

  Enter the values "1 June 2002" and "14​:00​:00" (when DST is in effect)

  Click "OK" to save these settings and dismiss the applet

  Open a Comment Prompt and type the following commands​:

  cd \Temp
  echo Hello > test.txt

  Now bring the "Date and Time Properties" applet back up, change the
  values to "1 November 2002" and "14​:00​:00" (when DST is no longer in
  effect) and click "OK" to save it

  Type the following command in the Command Prompt​:

  perl -e "print +(stat 'test.txt')[9]"

  This outputs a number like​:

  1022932805

  depending on how soon you created the file 'test.txt' with the "echo"
  command above after having set the time

  Finally, bring the "Date and Time Properties" applet back up,
  deselect the option to "Automatically adjust clock for daylight saving
  changes" and click "OK" to save it

  Repeat the above "perl" command in the Command Prompt, and a different
  number is output, in this case​:

  1022936405

The two values are different by 3600 seconds, i.e. the amount that a DST
adjustment was made for. The last modification time of the file thus
cannot be determined since there is no way (?) to programmatically
determine whether or not the "Automatically adjust ..." setting is
selected or not.

I believe that this output is wrong. The value returned by stat() should
be the same regardless of whether the "Automatically adjust ..." setting
is selected or not.

One final note​: stat() behaves as I would expect, i.e. always returning
the same value, on Windows 95/98/ME. It is only the "NT" type systems
(NT4/2000/XP) that have the problem.

---

Flags​:
  category=core
  severity=medium

---

Site configuration information for perl v5.8.0​:

Configured by radan at Thu Sep 5 09​:10​:23 2002.

Summary of my perl5 (revision 5 version 8 subversion 0) configuration​:
  Platform​:
  osname=MSWin32, osvers=4.0, archname=MSWin32-x86-perlio
  uname=''
  config_args='undef'
  hint=recommended, useposix=true, d_sigaction=undef
  usethreads=undef use5005threads=undef useithreads=undef
usemultiplicity=undef
  useperlio=define d_sfio=undef uselargefiles=undef usesocks=undef
  use64bitint=undef use64bitall=undef uselongdouble=undef
  usemymalloc=y, bincompat5005=undef
  Compiler​:
  cc='cl', ccflags ='-nologo -Gf -W3 -MD -DNDEBUG -O1 -DWIN32
-D_CONSOLE -DNO_STRICT -DHAVE_DES_FCRYPT -DUSE_PERLIO
-DPERL_MSVCRT_READFIX',
  optimize='-MD -DNDEBUG -O1',
  cppflags='-DWIN32'
  ccversion='', gccversion='', gccosandvers=''
  intsize=4, longsize=4, ptrsize=4, doublesize=8, byteorder=1234
  d_longlong=undef, longlongsize=8, d_longdbl=define, longdblsize=10
  ivtype='long', ivsize=4, nvtype='double', nvsize=8, Off_t='off_t',
lseeksize=4
  alignbytes=8, prototype=define
  Linker and Libraries​:
  ld='link', ldflags ='-nologo -nodefaultlib -release
-libpath​:"C​:\perl5\lib\CORE" -machine​:x86'
  libpth=D​:\PROGRA1\MICROS1\VC98\lib "D​:\openssl\lib"
  libs=libeay32.lib oldnames.lib kernel32.lib user32.lib gdi32.lib
winspool.lib comdlg32.lib advapi32.lib shell32.lib ole32.lib
oleaut32.lib netapi32.lib uuid.lib wsock32.lib mpr.lib winmm.lib
version.lib odbc32.lib odbccp32.lib msvcrt.lib
  perllibs=libeay32.lib oldnames.lib kernel32.lib user32.lib
gdi32.lib winspool.lib comdlg32.lib advapi32.lib shell32.lib ole32.lib
oleaut32.lib netapi32.lib uuid.lib wsock32.lib mpr.lib winmm.lib
version.lib odbc32.lib odbccp32.lib msvcrt.lib
  libc=msvcrt.lib, so=dll, useshrplib=yes, libperl=perl58.lib
  gnulibc_version='undef'
  Dynamic Linking​:
  dlsrc=dl_win32.xs, dlext=dll, d_dlsymun=undef, ccdlflags=' '
  cccdlflags=' ', lddlflags='-dll -nologo -nodefaultlib -release
-libpath​:"C​:\perl5\lib\CORE" -machine​:x86'

Locally applied patches​:
 

---

@​INC for perl v5.8.0​:
  C​:/perl5/lib
  C​:/perl5/site/lib
  .

---

Environment for perl v5.8.0​:
  HOME (unset)
  LANG (unset)
  LANGUAGE (unset)
  LD_LIBRARY_PATH (unset)
  LOGDIR (unset)
 
PATH=C​:\WINDOWS\system32;C​:\WINDOWS;C​:\WINDOWS\System32\Wbem;C​:\perl5\bin
  PERL5LIB (unset)
  PERL_BADLANG (unset)
  SHELL (unset)

[p5pRT]

From @steve-m-hay

My original bugreport specified

  category=core
  severity=medium

but the RT system seems to have logged it as

  category=unknown
  severity=low

instead. Is this a bug in perlbug?

[p5pRT]

From @eserte

Maybe some Windows expert may take a look at

  http​://www.codeproject.com/datetime/dstbugs.asp

and implement a fix based on the suggestions on this side. Otherwise
there should at least be a comment in perlport.pod.

Regards,
  Slaven

[p5pRT]

From @steve-m-hay

Regarding the above bug report​:

Is this something that is likely to be fixed in the Perl core any time soon?

Slaven Rezic found a website with an excellent description of the
problem, and a general solution to it, at the URL​:
http​://www.codeproject.com/datetime/dstbugs.asp

I would love to see this done in the Perl core, but I'd be way out of my
depth attempting it myself.

I could, however, produce a Win32​::* module based on it quite easily.
If there is no prospect of a core fix then I'd like to go ahead with
the module.

Any comments?

Steve

[p5pRT]

From @nwc10

On Mon, May 19, 2003 at 04​:44​:20PM +0100, Steve Hay wrote​:

Regarding the above bug report​:

Is this something that is likely to be fixed in the Perl core any time soon?

Not obviously. People who understand the core usually already have too
much on their plate, so unless someone is being bitten by this problem,
or considers it sufficiently serious to drop an existing pet project in
favour of it, it's unlikely.

Slaven Rezic found a website with an excellent description of the
problem, and a general solution to it, at the URL​:
http​://www.codeproject.com/datetime/dstbugs.asp

I would love to see this done in the Perl core, but I'd be way out of my
depth attempting it myself.

I could, however, produce a Win32​::* module based on it quite easily.
If there is no prospect of a core fix then I'd like to go ahead with
the module.

Any comments?

I don't use Win32, so I have no direct concern about how this is fix.

I think that the best thing to do is to write a module to implement the
fix (no idea about name), even if the module is superceded with a
core patch. Firstly because it's easier to experiment with behaviour
in perl rather than in C (even for people who are familiar with the
core) and secondly because the existing Perl installations that suffer
from this MS design bug would benefit from a module immediately.

From an open-source cat herding perspective, if you're able to contribute
a working perl module to implement a fix, then it's more likely that someone
will help you out by converting your perl algorithm into C in the core.
Offering patches has this strange effect of galvanising other people into
action.

Nicholas Clark

[p5pRT]

From @steve-m-hay

Nicholas Clark wrote​:

On Mon, May 19, 2003 at 04​:44​:20PM +0100, Steve Hay wrote​:

Regarding the above bug report​:

Is this something that is likely to be fixed in the Perl core any time soon?

Not obviously.
[snip]

Slaven Rezic found a website with an excellent description of the
problem, and a general solution to it, at the URL​:
http​://www.codeproject.com/datetime/dstbugs.asp
[snip]
I could, however, produce a Win32​::* module based on it quite easily.
If there is no prospect of a core fix then I'd like to go ahead with
the module.

Any comments?

I don't use Win32, so I have no direct concern about how this is fix.

I think that the best thing to do is to write a module to implement the
fix (no idea about name), even if the module is superceded with a
core patch.
[snip]

OK, I'll start putting together a Perl module.

Does anyone have any thoughts on a suitable name for it? How about
Win32​::UTCFileTime?

I was thinking of having the module provide overrides for the built-in
stat() and lstat() functions. By default, it would export the overrides
into CLORE​::GLOBAL. Explicitly asking for one/both functions would
disable the CORE​::GLOBAL override and only export the requested
function(s) to the caller​:

  # Override stat() and lstat() globally​:
  use Win32​::UTCFileTime;

  # Override stat() locally; don't override lstat().
  use Win32​::UTCFileTime qw(stat);

Why would anyone want to override stat() but not lstat()? Maybe​:

  # Override stat() and lstat() locally​:
  use Win32​::UTCFileTime qw(​:local);

makes more sense.

Any thoughts on these ideas, or any other ideas?

Steve

[p5pRT]

From @rgs

Steve Hay <steve.hay@​uk.radan.com> wrote​:

I was thinking of having the module provide overrides for the built-in
stat() and lstat() functions. By default, it would export the overrides
into CLORE​::GLOBAL.

As a general rule, I don't think that's a sensible default.
If people really mean to do require a global override, I'd
prefer an explicit '​:globally' tag (à la File​::Glob).

[p5pRT]

From @steve-m-hay

The uploaded file

  Win32-UTCFileTime-1.00.tar.gz

has entered CPAN as

file​: $CPAN/authors/id/S/SH/SHAY/Win32-UTCFileTime-1.00.tar.gz
size​: 28762 bytes
  md5​: 5ad0ba315e7fb2babf513b856b71834a

This module addresses this bug report.

It provides replacement stat(), lstat() and utime() functions that
correctly handle UTC file times (last access time, last modification
time and creation time). Perl's built-in functions of the same name
inherit some curious behaviour from the Microsoft C library which causes
the times to vary by one hour across DST season changes on NTFS filesystems.

Hopefully the workaround in this module will be incorporated into the
Perl core one day, but in the mean time this module should plug the gap.

Steve

[p5pRT]

From @smpeters

On Thu Jun 05 02​:10​:02 2003, shay wrote​:

The uploaded file

Win32-UTCFileTime-1.00.tar.gz

has entered CPAN as

file​: $CPAN/authors/id/S/SH/SHAY/Win32-UTCFileTime-1.00.tar.gz
size​: 28762 bytes
md5​: 5ad0ba315e7fb2babf513b856b71834a

This module addresses this bug report.

It provides replacement stat(), lstat() and utime() functions that
correctly handle UTC file times (last access time, last modification
time and creation time). Perl's built-in functions of the same name
inherit some curious behaviour from the Microsoft C library which causes
the times to vary by one hour across DST season changes on NTFS
filesystems.

Hopefully the workaround in this module will be incorporated into the
Perl core one day, but in the mean time this module should plug the gap.

Steve

Steve,

Do you know if there has been a fix in the MS core libraries since this
bug report?

Steve Peters

[p5pRT]

From @steve-m-hay

Steve Peters via RT wrote​:

On Thu Jun 05 02​:10​:02 2003, shay wrote​:

The uploaded file

Win32-UTCFileTime-1.00.tar.gz

has entered CPAN as

file​: $CPAN/authors/id/S/SH/SHAY/Win32-UTCFileTime-1.00.tar.gz
size​: 28762 bytes md5​: 5ad0ba315e7fb2babf513b856b71834a

This module addresses this bug report.

It provides replacement stat(), lstat() and utime() functions that
correctly handle UTC file times (last access time, last modification
time and creation time). Perl's built-in functions of the same name
inherit some curious behaviour from the Microsoft C library which
causes the times to vary by one hour across DST season changes on
NTFS filesystems.

Hopefully the workaround in this module will be incorporated into the
Perl core one day, but in the mean time this module should plug the
gap.

Steve

Steve,

Do you know if there has been a fix in the MS core libraries since
this bug report?

Steve Peters

I have a perl built with VC++ 2005 (VC8) and it still exhibits the same
behaviour reported in this bug when run on Windows Vista.

That's what I'd expect, really. When I created the Win32-UTCFileTime
module to workaround this bug, I found the suggestion that this is all
intended behaviour on Microsoft's part, i.e. they don't regard it as a
bug, so it's not surprising that it isn't fixed.

[p5pRT]

From @nwc10

On Fri, Nov 16, 2007 at 02​:52​:25PM -0000, Steve Hay wrote​:

Steve Peters via RT wrote​:

On Thu Jun 05 02​:10​:02 2003, shay wrote​:

It provides replacement stat(), lstat() and utime() functions that
correctly handle UTC file times (last access time, last modification
time and creation time). Perl's built-in functions of the same name
inherit some curious behaviour from the Microsoft C library which
causes the times to vary by one hour across DST season changes on
NTFS filesystems.

That's what I'd expect, really. When I created the Win32-UTCFileTime
module to workaround this bug, I found the suggestion that this is all
intended behaviour on Microsoft's part, i.e. they don't regard it as a
bug, so it's not surprising that it isn't fixed.

Sigh. There are various things I could write about Microsoft for this one,
but most would cause either the police to arrest me for attempting to incite
homicide, or Kermit's lawyers to issue a write for libel.

DST is a whim of politicians. That should be a good enough reason to avoid it.
Look closer, and you see that if you have DST, then for one hour, wallclock
times repeat. Which 1​:30am on that particular Sunday was it?

And then timezones themselves. If I have a laptop, and it goes on a trip
with me to another continent or country or state or county just a Native
American Reservation...

Nicholas Clark

PS Should Perl 6 declare sanity on this from release 6.0.0 ?

[p5pRT]

@dcollinsn - Status changed from 'open' to 'stalled'