Differences between revisions 10 and 11
Revision 10 as of 2017-03-17 10:52:31
Size: 5820
Editor: 130
Comment:
Revision 11 as of 2017-04-17 12:19:00
Size: 5965
Editor: 82-181-83-123
Comment:
Deletions are marked like this. Additions are marked like this.
Line 67: Line 67:
To be done. {{{
df = DataFrame
df.index = pd.to_datetime(df['timestamp'], unit='s', utc=True)
df.index = df.index.tz_convert('Europe/Helsinki') # or TZ object
}}}

This is a table of common time functions in Python. Names are listed as if this has been run first: from time import * ; from datetime import datetime,date,time. Inputs in parentheses signify the function is a method on that object.

There are two ways of dealing with timezones:

  • Lightweight / system: the operating system has functions for UTC and localtime timezones. These are reflected below, in [utc] or [local]. A process can only have one timezone, and it is set by an environment variable. To set it

    • Set the variable before starting the process, in bash you can do this with TZ=Europe/Helsinki python file.py .... This can be good for testing different time zones.

    • Within python: import os ; os.environ['TZ'] = 'Europe/Helsinki' ; time.tzset() . The last function is the key one!

    • There is only one timezone per process.
    • search "tz database" for list of timezone info. This is considered the "truth" about timezones, and includes all info since 1970 for every point on earth, including all the random political changes, such as changing laws on summer time.
  • tzinfo of datetime objects: Use advanced functionality of a separate module like pytz + timezone support in python. This gets difficult, and is usually not needed. This involves using the tzinfo argument in the datetime module. A datetime object with a tzinfo is called an "aware" datetime object, and knows its timezone, so can usefully convert itself to unixtime without just relying on local vs utc of step 1.

What is a timezone? unixtime unambiguously refers to a single point in time on earth. Timezone does not affect unix time. It is universal. unixtime+timezone=localtime. Also, you could say that unixtime=linarization of utc. A generally easy strategy is to take all inputs (knowing the timezone) and convert it to unixtime. This is an easy time scale to work with, for example this allows you to do arithmetic, such as "how many seconds apart are these two events?". Thanks to the magic of the tz database, everything will work, considering things like summer time. You don't need to consider timezones until you need to output time again, and then you use the timezone to format it. If you need something such as hour or weekday, that becomes an extra tag, which you get from localtime and keep in addition to the unixtime. (If you don't need to do arithmetic on time, you can skip the unixtime part perhaps). If you need to find something like "the same time of day, but in 28 days", then you would convert to datetime, add timedelta(days=28), and then convert back to unixtime. Basically, timezones are extremely arbitrary, and if you need a linear scale, use unixtime, if you need "human perceived time of day", use datetime objects or time structs.

Using the proper techniques on this page can handle all time math related problems, except leap seconds.

System based timezone handling

The following chart only considers the first option (lightweight) for timezone handling above (it does *not* consider the tzinfo argument for any functions (naive vs aware datetime objects, the second and harder option for dealing with timezones))

Function

input

output

asctime

struct

string

ctime

unix

string

gmtime

unix

struct[utc]

localtime

unix

struct[local]

mktime

struct[local]

unix

DST flag important????

strftime

struct

string

ref

strptime

string

struct

time

-

unix

calendar.timegm

struct[utc]

unix

datetime/date.fromtimestamp

unix

datetime.date[local]

datetime.utcfromtimestamp

unix

datetime[utc]

datetime/date.fromordinal

days since 0001-01-01

datetime.date

date.timetuple

(date)

struct

datetime.timetuple

(datetime)

struct

no timezone conversion

datetime.utctimetuple

(datetime)

struct[utc]

requires tz-awere dt to convert

datetime.toordinal

(datetime)

days since 0001-01-01

datetime.isoformat

(datetime)

string in YYYY-MM-DDTHH:MM...

datetime.ctime

(datetime)

string

datetime.strftime

(datetime)

string

ref

time.isoformat

(time)

string

time.mktime(dt.timetuple())

(datetime)[local]

unix

no easy way to do this on python2

datetime.timestamp (Python 3)

datetime[local or aware]

unix

Python based timezone handling

This section is incomplete.

The above section Python functions use the C library to convert between. There are only two classes of datetimes: UTC and localtime. Timezone is determined by the system. Things are confusing because each datetime object has no concept of if it is UTC or localtime, you have to track that yourself.

Basic definition: a "naive" datetime object has no timezone associated with it (dt.tzinfo=None). This means that you don't actually know when the time is! You can't convert it to any other timezone until you say what the starting time is. This is only useful if all datetimes are in this same timezone. An "aware" datetime object has a .tzinfo attribute that tells the timezone. This allows you to convert to other timezones, including unixtime.

timezone = pytz.timezone('Europe/Helsinki')
dt = datetime.datetime.today()

# naive -> aware (Add a timezone to a datetime that has none)
dt_aware = timezone.localize(dt)

# unixtime -> aware datetime 
datetime.datetime.fromtimestamp(unixtime, self._timezone)

Pandas

df = DataFrame
df.index = pd.to_datetime(df['timestamp'], unit='s', utc=True)
df.index = df.index.tz_convert('Europe/Helsinki')   # or TZ object

DebianNotes/PythonTime (last edited 2017-04-17 12:19:00 by 82-181-83-123)