Convert pubdate in RSS metadata

[theotherbassist]

I know this has been asked a lot around the web for other platforms, but I haven't seen much pubdate frustration in the context of livecode.

RSS feed data, as we know, is formatted as XML. The corresponding date/time format for entries--filed under the node "pubdate"--looks like Mon, 11 Apr 2016 15:00:00 +0100 (internet date) but often sources specify a timezone abbreviation (e.g. "GMT" or "EST") instead of the timezone offset number. Yes, it's technically incorrect to do this, but it's common practice so there's no avoiding it.

Does anyone know a simple method of "standardizing" publication dates/times when they vary by timezone, and especially when the TZ offset isn't numerical? I don't really want to have to use an API call to google maps or something similar, but I also don't want to have to code in the whole globe and its timezone abbreviations with all the associated DST dates/times just to sort articles by the order they come in.

Am I missing something obvious?

[FourthWorld]

Welcome to the wacky world of Internet specs, where everytime someone coughs another RFC describing a new time format appears.

In a perfect world everyone would either use seconds in Unix time or RFC 2822, which LC uses for "internet date". But alas, ours is not a perfect world.

I got so frustrated coming up with a perfect solution that I resorted to an imperfect one: I found the following lookup table somewhere on the Web and incorporated it as a custom property to do the translation to Internet time. The weakness is that some of those times may be off by an hour, as daylight savings/summer hours/whatever else they may be called regionally come into play. But in my case it was for a non-mission-critical app in which an hour's difference isn't likely to matter much if noticed at all.

For the long term there's a project spec'd for the core team to build a framework for handling a great many of the myriad "standard" time formats. But given the priorities for v8 I wouldn't expect it soon, so unless you find a better solution (and please share if you do) this is what I'm using at the moment:

Code: Select all

A +0100
ADT -0300
ADT -0300
AFT +0430
AKDT -0800
AKST -0900
ALMT +0600
AMST -0300
AMT +0400
AMT -0400
ANAST +1200
ANAT +1200
AQTT +0500
ART -0300
AST +0300
AST -0400
AST -0400
AST -0400
AZOST 0000
AZOT -0100
AZST +0500
AZT +0400
B +0200
BNT +0800
BOT -0400
BRST -0200
BRT -0300
BST +0600
BST +0100
BTT +0600
C +0300
CAST +0800
CAT +0200
CCT +0630
CDT +1030
CDT -0400
CDT -0500
CEST +0200
CET +0100
CET +0100
CHADT +1345
CHAST +1245
CKT -1000
CLST -0300
CLT -0400
COT -0500
CST +0800
CST +0930
CST -0600
CST -0500
CST -0600
CVT -0100
CXT +0700
ChST +1000
D +0400
DAVT +0700
E +0500
EASST -0500
EAST -0600
EAT +0300
EAT +0300
ECT -0500
EDT +1100
EDT -0400
EDT -0400
EDT +1100
EEST +0300
EEST +0300
EEST +0300
EET +0200
EET +0200
EET +0200
EGT -0100
EST +1000
EST -0500
EST -0500
EST -0500
ET -0500
ET -0500
ET -0500
F +0600
FJST +1300
FJT +1200
FKST -0300
FKT -0400
FNT -0200
G +0700
GALT -0600
GAMT -0900
GET +0400
GFT -0300
GILT +1200
GMT 0000
GST +0400
GYT -0400
H +0800
HAA -0300
HAA -0300
HAC -0500
HADT -0900
HAE -0400
HAE -0400
HAP -0700
HAR -0600
HAST -1000
HAT -0230
HAY -0800
HKT +0800
HLV -0430
HNA -0400
HNA -0400
HNA -0400
HNC -0600
HNC -0600
HNE -0500
HNE -0500
HNE -0500
HNP -0800
HNR -0700
HNT -0330
HNY -0900
HOVT +0700
I +0900
ICT +0700
IDT +0300
IOT +0600
IRDT +0430
IRKST +0900
IRKT +0900
IRST +0330
IST +0200
IST +0530
IST +0100
JST +0900
K +1000
KGT +0600
KRAST +0800
KRAT +0800
KST +0900
KUYT +0400
L +1100
LHDT +1100
LHST +1030
LINT +1400
M +1200
MAGST +1200
MAGT +1200
MART -0930
MAWT +0500
MDT -0600
MESZ +0200
MEZ +0100
MHT +1200
MMT +0630
MSD +0400
MSK +0400
MST -0700
MUT +0400
MVT +0500
MYT +0800
N -0100
NCT +1100
NDT -0230
NFT +1130
NOVST +0700
NOVT +0600
NPT +0545
NST -0330
NUT -1100
NZDT +1300
NZDT +1300
NZST +1200
NZST +1200
O -0200
OMSST +0700
OMST +0700
P -0300
PDT -0700
PET -0500
PETST +1200
PETT +1200
PGT +1000
PHOT +1300
PHT +0800
PKT +0500
PMDT -0200
PMST -0300
PONT +1100
PST -0800
PT -0800
PWT +0900
PYST -0300
PYT -0400
Q -0400
R -0500
RET +0400
S -0600
SAMT +0400
SAST +0200
SBT +1100
SCT +0400
SGT +0800
SRT -0300
SST -1100
T -0700
TAHT -1000
TFT +0500
TJT +0500
TKT +1300
TLT +0900
TMT +0500
TVT +1200
U -0800
ULAT +0800
UYST -0200
UYT -0300
UT -0800
UZT +0500
V -0900
VET -0430
VLAST +1100
VLAT +1100
VUT +1100
W -1000
WAST +0200
WAT +0100
WDT +0900
WEST +0100
WEST +0100
WESZ +0100
WET 0000
WEZ 0000
WFT +1200
WGST -0200
WGT -0300
WIB +0700
WIT +0900
WITA +0800
WST +0100
WST +0800
WST +1300
WT 0000
X -1100
Y -1200
YAKST +1000
YAKT +1000
YAPT +1000
YEKST 0000
YEKT 0000
Z 0000

[theotherbassist]

Ugh, you're confirming all my fears. I was afraid of having to make a table of time zones. Unfortunately, an rss reader hosting international feeds needs the ability to sort out pubdates down to the minute... including dst offsets. Maybe i'll scrap chronological sorting altogether. It seems so silly--it's such a simple thing. In the future, our machine overlords will give us a hard time about all this "daylight savings time" business.

Maybe a silly question, but why are there duplicates in your list? Does that refer to seasons or something?

[FourthWorld]

Ugh, you're confirming all my fears. I was afraid of having to make a table of time zones. Unfortunately, an rss reader hosting international feeds needs the ability to sort out pubdates down to the minute... including dst offsets. Maybe i'll scrap chronological sorting altogether. It seems so silly--it's such a simple thing. In the future, our machine overlords will give us a hard time about all this "daylight savings time" business.

The worst you can imagine with handling time issues is only the beginning:
https://www.youtube.com/watch?v=-5wpm-gesOY

Maybe a silly question, but why are there duplicates in your list? Does that refer to seasons or something?

Laziness squared: I just copied the list from an online source, so apparently they were too lazy to catch those and so was I.

[theotherbassist]

have seen that video... maybe an API call to google via feed location is the way to go after all

[jacque]

They aren't exactly duplicates all the time:
CST +0800
CST +0930
CST -0600
CST -0500
CST -0600

I'm thinking it must be intentional but there's no clue what it means.

[MaxV]

Did you see this livecode date library? https://github.com/derbrill/libdate

[theotherbassist]

They aren't exactly duplicates all the time:
CST +0800
CST +0930
CST -0600
CST -0500
CST -0600

I'm thinking it must be intentional but there's no clue what it means.

After a little thought, I realized something. Those are different timezones with overlapping abbreviations. "CST"= "Central Standard Time" in the US and "China Standard Time" on the other side of the globe. The first and second instance of each must represent the dst offset or have something to do with geographical variation within the zone.

[theotherbassist]

I also don't know why I didn't think this whole thing through a little bit more.

Locations change clock offsets with UTC, but timezones themselves have a constant offset. Because areas that observe DST are actually just switching timezones, the database method might actually work rather simply (until the abbreviation definitions change--but a mere rule change of DST implementation days wouldn't break anything so long as the news sources are updating their code).

When an area switches TZ, a newspaper in that area should switch its RSS feed's TZ label accordingly. If not, it's the source's fault that content isn't sorting correctly by date--not the app's. The program can only assume the data provided is correct without collaborating with outside sources. For example, Los Angeles switches back and forth between PST and PDT. But regardless of which is currently provided with the pubdates, "PST" and "PDT" have standard definitions.

Now when it comes to the overlap of actual abbreviated codes in practice, I suppose you would have to ask the user something like "is this feed located in America or China?" if pulling a test pubdate node reveals "CST." You could then swap in a provisional "CST(China)" for the ambiguous "CST" label before converting with your database.

What do you guys think?

[FourthWorld]

Given the weird inconsistencies and sometimes even blatant disregard for widely published specs I've seen in some of the wackier RSS feeds I've tested with, I believe there's little evidence that people generating them can be trusted to maintain even small technical details like using the proper time zone designation.

But there's only so much a consumer of such feeds can do, so it may be as good as it gets.