# do not edit -- automatically generated by arch changelog # arch-tag: automatic-ChangeLog--scott@netsplit.com--projects/planet--devel--0.0 # 2004-01-20 07:34:56 GMT Scott James Remnant patch-22 Summary: fix further bogocity with save_uri (actually save the redirected uri) Revision: planet--devel--0.0--patch-22 modified files: planetlib.py 2004-01-20 07:33:54 GMT Scott James Remnant patch-21 Summary: clean up and a couple of minor bug fixes (date_format not being used, save_uri set wrongly) Revision: planet--devel--0.0--patch-21 modified files: planet.py planetlib.py 2004-01-20 01:11:00 GMT Scott James Remnant patch-20 Summary: change cowardly debugs to warnings Revision: planet--devel--0.0--patch-20 modified files: planetlib.py 2004-01-20 01:02:32 GMT Scott James Remnant patch-19 Summary: ignore feeds containing obviously bogus dates Revision: planet--devel--0.0--patch-19 modified files: planetlib.py 2004-01-20 00:52:54 GMT Scott James Remnant patch-18 Summary: sync with jdub, we now have equal source trees Revision: planet--devel--0.0--patch-18 Patches applied: * jdub@perkypants.org--projects/planet--devel--0.0--patch-35 new version of feedparser * jdub@perkypants.org--projects/planet--devel--0.0--patch-36 synced with scott modified files: feedparser.py new patches: jdub@perkypants.org--projects/planet--devel--0.0--patch-35 jdub@perkypants.org--projects/planet--devel--0.0--patch-36 2004-01-19 01:22:33 GMT Scott James Remnant patch-17 Summary: reverse part of the last change. Revision: planet--devel--0.0--patch-17 modified files: feedparser.py 2004-01-19 01:15:05 GMT Scott James Remnant patch-16 Summary: Fix feedparser to work with atom mode=xml feeds Revision: planet--devel--0.0--patch-16 modified files: feedparser.py 2004-01-18 21:59:33 GMT Scott James Remnant patch-15 Summary: Invent (and cache) times for entries without them Revision: planet--devel--0.0--patch-15 modified files: planetlib.py 2004-01-18 21:13:20 GMT Scott James Remnant patch-14 Summary: catch up with hjdub Revision: planet--devel--0.0--patch-14 Patches applied: * jdub@perkypants.org--projects/planet--devel--0.0--patch-31 scott points out that htmltmpl can do encoding all by itself. gar. i've also set the default log_level to DEBUG -> i'm using the examples for testing all the time now, and it makes it easier when setting up. * jdub@perkypants.org--projects/planet--devel--0.0--patch-32 add content xmlns to rss10 example * jdub@perkypants.org--projects/planet--devel--0.0--patch-33 Scott's very excellent and generic changes while I was asleep * jdub@perkypants.org--projects/planet--devel--0.0--patch-34 mention the uri on 404 errors modified files: .arch-ids/planetlib.py.id cache/.arch-ids/=id examples/.arch-ids/index.html.tmpl.id output/.arch-ids/=id planetlib.py {arch}/planet/planet--devel/planet--devel--0.0/scott@netsplit.com--projects/patch-log/base-0 {arch}/planet/planet--devel/planet--devel--0.0/scott@netsplit.com--projects/patch-log/patch-1 {arch}/planet/planet--devel/planet--devel--0.0/scott@netsplit.com--projects/patch-log/patch-10 {arch}/planet/planet--devel/planet--devel--0.0/scott@netsplit.com--projects/patch-log/patch-11 {arch}/planet/planet--devel/planet--devel--0.0/scott@netsplit.com--projects/patch-log/patch-2 {arch}/planet/planet--devel/planet--devel--0.0/scott@netsplit.com--projects/patch-log/patch-3 {arch}/planet/planet--devel/planet--devel--0.0/scott@netsplit.com--projects/patch-log/patch-4 {arch}/planet/planet--devel/planet--devel--0.0/scott@netsplit.com--projects/patch-log/patch-5 {arch}/planet/planet--devel/planet--devel--0.0/scott@netsplit.com--projects/patch-log/patch-6 {arch}/planet/planet--devel/planet--devel--0.0/scott@netsplit.com--projects/patch-log/patch-7 {arch}/planet/planet--devel/planet--devel--0.0/scott@netsplit.com--projects/patch-log/patch-8 {arch}/planet/planet--devel/planet--devel--0.0/scott@netsplit.com--projects/patch-log/patch-9 modified directories: cache cache/.arch-ids output output/.arch-ids new patches: jdub@perkypants.org--projects/planet--devel--0.0--patch-31 jdub@perkypants.org--projects/planet--devel--0.0--patch-32 jdub@perkypants.org--projects/planet--devel--0.0--patch-33 jdub@perkypants.org--projects/planet--devel--0.0--patch-34 2004-01-13 00:17:33 GMT Scott James Remnant patch-13 Summary: merge with jdub again Revision: planet--devel--0.0--patch-13 modified files: examples/config.ini examples/rss10.xml.tmpl 2004-01-12 18:29:29 GMT Scott James Remnant patch-12 Summary: Catch exception on remote read, so we can fall back to subscribing from the cache Revision: planet--devel--0.0--patch-12 modified files: planetlib.py 2004-01-11 22:28:51 GMT Scott James Remnant patch-11 Summary: increase offset, oops Revision: planet--devel--0.0--patch-11 modified files: planetlib.py 2004-01-11 22:13:36 GMT Scott James Remnant patch-10 Summary: add +00:00 to the end of ISO dates Revision: planet--devel--0.0--patch-10 modified files: planetlib.py 2004-01-11 22:11:20 GMT Scott James Remnant patch-9 Summary: More timezone/offset fixing to make sure what's in the config file is the UTC offset Revision: planet--devel--0.0--patch-9 modified files: planetlib.py 2004-01-11 21:00:04 GMT Scott James Remnant patch-8 Summary: offset needs to be subtracted, not added Revision: planet--devel--0.0--patch-8 modified files: planetlib.py 2004-01-11 20:41:07 GMT Scott James Remnant patch-7 Summary: don't cache empty feeds Revision: planet--devel--0.0--patch-7 modified files: planetlib.py 2004-01-10 17:35:18 GMT Scott James Remnant patch-6 Summary: MERGE WARS, EPISODE IV Revision: planet--devel--0.0--patch-6 Mostly I've just changed the templates and things so someone could grab this source, edit config.ini and their Planet site pretty much runs out of the box. * Added a few more things to the [Planet] config section: link, owner_name and owner_email. These then get dropped into the template, and mean that the examples templates don't contain any references to Planet GNOME now. * output_dir changed to 'output' and both that and the empty cache directory included in the repository. * Drop cgi.escape() in favour of * Use name not title in opml.xml for each channel * Use id where a unique id is called for, such as rdf:resource, and link to actually give the HTML output. (id falls back to link) * If creating a Channel or subscribing it fails, don't break, just carry on regardless. * For both Items and the template itself, there's not just 'date' available (the display-formatted date) there's also date_iso and date_822 formatted appropriately. * New config option to give the display date format. * Fix bogus config.get() calls, they don't return the third option at all. * Some vague code tidying (I like 80 columns) * Drop reference to True and False new files: cache/.arch-ids/=id output/.arch-ids/=id modified files: Makefile examples/config.ini examples/foafroll.xml.tmpl examples/index.html.tmpl examples/opml.xml.tmpl examples/rss10.xml.tmpl examples/rss20.xml.tmpl planet.py planetlib.py new directories: cache cache/.arch-ids output output/.arch-ids 2004-01-10 16:34:46 GMT Scott James Remnant patch-5 Summary: Merge jdub's changes Revision: planet--devel--0.0--patch-5 Patches applied: * jdub@perkypants.org--projects/planet--devel--0.0--patch-17 clean target * jdub@perkypants.org--projects/planet--devel--0.0--patch-18 merged with scott's new planet code * jdub@perkypants.org--projects/planet--devel--0.0--patch-19 make it cleaner * jdub@perkypants.org--projects/planet--devel--0.0--patch-20 python 2.2 compatibility * jdub@perkypants.org--projects/planet--devel--0.0--patch-21 forced delete, yucky but hey * jdub@perkypants.org--projects/planet--devel--0.0--patch-22 Say we need Python 2.2, include Planet name in templates, add encoded_content to NewsItem template (and update example), add an RSS 2.0 template example * jdub@perkypants.org--projects/planet--devel--0.0--patch-23 hfsnw, multiple outputs rock * jdub@perkypants.org--projects/planet--devel--0.0--patch-24 saner example template names, add an info log about template filename * jdub@perkypants.org--projects/planet--devel--0.0--patch-25 template fixes * jdub@perkypants.org--projects/planet--devel--0.0--patch-26 fix up rss 1.0 template * jdub@perkypants.org--projects/planet--devel--0.0--patch-27 opml output template * jdub@perkypants.org--projects/planet--devel--0.0--patch-28 bad xml in opml example... gar. * jdub@perkypants.org--projects/planet--devel--0.0--patch-29 opml is so much teh suck * jdub@perkypants.org--projects/planet--devel--0.0--patch-30 foafroll template, for foaf/rdf blogroll new files: compat_logging/.arch-ids/=id compat_logging/.arch-ids/__init__.py.id compat_logging/.arch-ids/config.py.id compat_logging/.arch-ids/handlers.py.id compat_logging/__init__.py compat_logging/config.py compat_logging/handlers.py examples/.arch-ids/foafroll.xml.tmpl.id examples/.arch-ids/opml.xml.tmpl.id examples/.arch-ids/rss10.xml.tmpl.id examples/.arch-ids/rss20.xml.tmpl.id examples/foafroll.xml.tmpl examples/opml.xml.tmpl examples/rss10.xml.tmpl examples/rss20.xml.tmpl modified files: Makefile examples/config.ini examples/index.html.tmpl planet.py planetlib.py renamed files: examples/.arch-ids/planet.tmpl.id ==> examples/.arch-ids/index.html.tmpl.id examples/planet.tmpl ==> examples/index.html.tmpl new directories: compat_logging compat_logging/.arch-ids modified directories: {arch}/planet/planet--devel/planet--devel--0.0/scott@netsplit.com--projects {arch}/planet/planet--devel/planet--devel--0.0/scott@netsplit.com--projects/patch-log new patches: jdub@perkypants.org--projects/planet--devel--0.0--patch-17 jdub@perkypants.org--projects/planet--devel--0.0--patch-18 jdub@perkypants.org--projects/planet--devel--0.0--patch-19 jdub@perkypants.org--projects/planet--devel--0.0--patch-20 jdub@perkypants.org--projects/planet--devel--0.0--patch-21 jdub@perkypants.org--projects/planet--devel--0.0--patch-22 jdub@perkypants.org--projects/planet--devel--0.0--patch-23 jdub@perkypants.org--projects/planet--devel--0.0--patch-24 jdub@perkypants.org--projects/planet--devel--0.0--patch-25 jdub@perkypants.org--projects/planet--devel--0.0--patch-26 jdub@perkypants.org--projects/planet--devel--0.0--patch-27 jdub@perkypants.org--projects/planet--devel--0.0--patch-28 jdub@perkypants.org--projects/planet--devel--0.0--patch-29 jdub@perkypants.org--projects/planet--devel--0.0--patch-30 2004-01-10 05:31:55 GMT Scott James Remnant patch-4 Summary: Try/Except in case of failure Revision: planet--devel--0.0--patch-4 modified files: planet.py 2004-01-10 05:13:33 GMT Scott James Remnant patch-3 Summary: merge jdub's feedparser update Revision: planet--devel--0.0--patch-3 Patches applied: * jdub@perkypants.org--projects/planet--devel--0.0--patch-16 feedparser.py 2.7.1 modified files: feedparser.py new patches: jdub@perkypants.org--projects/planet--devel--0.0--patch-16 2004-01-10 04:52:34 GMT Scott James Remnant patch-2 Summary: Don't need global_vars here, there aren't any Revision: planet--devel--0.0--patch-2 modified files: planet.py 2004-01-10 04:36:29 GMT Scott James Remnant patch-1 Summary: complete rewrite, the "Beagle 2" release Revision: planet--devel--0.0--patch-1 This is what happens when I've got nothing better to do on a Friday evening/night. I started out to see how easy it would be to add ETag support to the existing code, and ended up pretty much rewriting the whole thing top to bottom. So settle back, get a drink, and let me tell you all about this new version which I'm naming the "Beagle 2" release. I'll start with the "wow, cool" stuff and work my way down to the "oh, right" stuff. I should note that you can't simply drop this code in and use your old Planet config, I got a bit over-zealous and changed every single config option and template variable. Sorry, I probably shouldn't have said to get that drink; you might want to clean up a bit before carrying on. * We now have ETag and If-Modified-Since support. In simple terms, this means that if a feed hasn't changed then it won't be downloaded again until it does. * This means the 'data_directory' has been changed to the 'cache_directory' and is now called 'cache' by default. * And in fact, how it *uses* this directory has been completely changed. The old code was pretty damned sucky in fact, it had to go. This is what it used to do: - Download the feed for each blog, apply the template in entry.tmpl to it then write the result into a file in the data directory named after the link. - Grab all the files in the data directory, sort them by mtime and build the page up from this (using page.tmpl). There was no link between the feed and the entries, whatever was in the data directory would get displayed. That's why we get duplicated entries when people changed their link. This is what we now do: - Download the feed for each blog and save the unparsed feed into the cache directory, named after the uri we got it from. - Parse the feed and aggregate them all in memory, applying the template to the lot. Actually that's not quite true, when we create the channel we load the cached feed, etag and modified information, *then* we download the feed if it has changed. We parse whichever won. * This means I changed the templates too, we now only have one called 'planet.tmpl' (in examples, anyway) -- you basically paste the old 'entry.tmpl' inside the new loop. The new single config file variable is therefore 'template_file' and for kicks the output file is now 'output_file'. * I suppose now's the time to rave about the new code in 'planetlib.py'. This consists of three classes, two of which are loosely based on the old ones from 'planet.py'. Planet: A collection of channels (a website). Channel: A collection of news items (a parsed feed). NewsItem: A news item. Here's a quick use of this: import planetlib p = planetlib.Planet() p.subscribe(planetlib.Channel("http://somewhere.com/rss")) p.subscribe(planetlib.Channel("http://elsewhere.org/atom")) for channel in p.channels(): # Make the blogroll for item in p.items(): # And make the web page I'm sure you already realise how easy this new API will be to import blogrolls and export RDF and blogrolls ;) * This obviously means there's not much left in 'planet.py' now, in fact all it does is parse the config file, use the planetlib API to build up a planet, then use the template stuff to write it back out. Btw, I changed 'blog_items_to_show' to the slightly more sensible 'items_per_page' (because it's easy to have multiple pages now :p) * I took this opportunity to make the template stuff a bit smarter too. You have two TMPL_LOOP variables 'Items' and 'Channels' and a TMPL_VAR called 'date' (was 'freshness') available. Inside TMPL_LOOP Channels you have 'uri' (the feed, not the link), 'title', 'description' and 'link' available, taken from the feed. You ALSO have any config option you left in the configuration file, such as 'name', 'face', 'facewidth', etc. (as a Brucey-Bonus, 'name' defaults to 'title'). Inside TMPL_LOOP Items you have 'id', 'date', 'title', 'summary', 'content' and 'link'; all referring to the news item itself. You also have 'new_date' (was 'new_day' and 'date') which is empty unless this entry starts on a new date (TMPL_IF it, in fact you can TMPL_IF anything now). And that's not all, you ALSO have everything from the channel but prefixed with 'channel_' (so 'channel_name', 'channel_face', 'channel_title', etc.) * Because of this new feature of anything in the config file being available in the template, I'm forcing the proper use of the [DEFAULT] section. Everything that was there is now in the [Planet] section (I renamed nearly all of them anyway :p), the [DEFAULT] section provides defaults for the blogs... For Planet Debian, I can go: [DEFAULT] facewidth = 64 faceheight = 64 [http://www.livejournal.com/users/keybuk/data/rss] name = Scott James Remnant face = keybuk.png facewidth = 51 # faceheight defaults to 64 * Planet Debian moved to a debian/unstable box, so I have Python 2.3 goodness available, hurrah! Dropping the newer sgmllib in was what was coredumping it *sulk*. I've taken advantage of this and dropped log4py in favour of the new 'logging' module. 'log_level' is now therefore one of DEBUG, INFO, WARNING, ERROR or CRITICAL. * Also gone is SimpleStripper now we don't use it. * I've written a new example configuration file and template file for the 'examples' directory. * 'roll_in_utc' has been lost because feedparser always returns UTC. Whew, not bad for an evening's work. new files: .arch-ids/planetlib.py.id examples/.arch-ids/planet.tmpl.id examples/planet.tmpl planetlib.py removed files: .arch-ids/SimpleStripper.py.id .arch-ids/log4py.py.id SimpleStripper.py examples/.arch-ids/entry.tmpl.id examples/.arch-ids/page.tmpl.id examples/entry.tmpl examples/page.tmpl log4py.py modified files: AUTHORS examples/config.ini planet.py 2004-01-09 21:38:38 GMT Scott James Remnant base-0 Summary: tag of jdub@perkypants.org--projects/planet--devel--0.0--patch-15 Revision: planet--devel--0.0--base-0 (automatically generated log message) new patches: jdub@perkypants.org--projects/planet--devel--0.0--base-0 jdub@perkypants.org--projects/planet--devel--0.0--patch-1 jdub@perkypants.org--projects/planet--devel--0.0--patch-2 jdub@perkypants.org--projects/planet--devel--0.0--patch-3 jdub@perkypants.org--projects/planet--devel--0.0--patch-4 jdub@perkypants.org--projects/planet--devel--0.0--patch-5 jdub@perkypants.org--projects/planet--devel--0.0--patch-6 jdub@perkypants.org--projects/planet--devel--0.0--patch-7 jdub@perkypants.org--projects/planet--devel--0.0--patch-8 jdub@perkypants.org--projects/planet--devel--0.0--patch-9 jdub@perkypants.org--projects/planet--devel--0.0--patch-10 jdub@perkypants.org--projects/planet--devel--0.0--patch-11 jdub@perkypants.org--projects/planet--devel--0.0--patch-12 jdub@perkypants.org--projects/planet--devel--0.0--patch-13 jdub@perkypants.org--projects/planet--devel--0.0--patch-14 jdub@perkypants.org--projects/planet--devel--0.0--patch-15 jdub@perkypants.org--projects/planet--pyblagg--0.0--base-0