Content tagged with [syndication] on DuncanMackenzie.net

Tracking RSS Feed Statistics

Duncan Mackenzie — Wed, 05 Sep 2007 07:58:00 GMT

A few of my friends and co-workers have jumped on the FeedBurner bandwagon, and I've been pretty impressed with the stats they've been gathering. I'm not willing to send my readers off to someone else's service though (even though there are some reasonably safe ways to do that), so I started to think about how best to gather similar info myself. One idea would be to ship the IIS logs from my site back to my PC on a regular basis and run them through a LogParser script that would figure out all the stats for me... but that seems like a rather manual (or difficult to automate) and data transfer intensive method.

Instead, what I decided to do was to log every feed requests into a table on my SQL Server (actual SQL updates occur at regular intervals, not on every feed request). I log the feed URL being requested, the user agent of the requester, the IP Address of the requester, the number of subscribers represented by the user agent string (Google's feed reader, amongst others, indicates the # of subscribers in the user agent string) and the date/time they made the request.

What happens next is currently a manual process, but I'm working on making it automatic...

For a given 24-hour period:

I sum up the number of requests by user agent/IP address combo
I filter down to user agent/IP address combos that made more than one request to the same feed during that time period... this is intended to distinguish between a manual user visit and an RSS aggregator. (of course, if an RSS aggregator is set to be very polite and only hit my site once per day, then this will cut those #s out of the list)
I sum up the average subscriber # for all useragent/IP address combos ... to avoid counting any single aggregator user more than once and also to accommodate any fluctuation in web aggregator subscriber counts throughout the day.

This produces a final # that I believe is roughly accurate... 792 (for the past 24 hours) across all of my feeds (mostly either the main feed or the Visual Basic feed).

About half of the requests (244 total) filtered out because they made only one request in 24 hours do appear to be aggregators (looking at user agent strings) so that would add 120 or so to this #, but in the interest of not inflating the numbers, I think I'll stick with the calculations I've worked out so far. My other worry is that some of the web-based aggregators might be producing some duplicate values due to multiple source IP addresses, probably as a result of having aggregation code running on multiple nodes of a web farm.

Has anyone else tried to implement their own FeedBurner-style stats? Any thoughts on my logic so far?

Cool gadget using sparklines to show your RSS stats from Feedburner

Duncan Mackenzie — Mon, 07 May 2007 06:30:00 GMT

A friend of mine has written this google gadget in true mashup style... it uses an API from Feedburner, a service to produce the graphics, and works async to update itself right in the browser ... and produces a sweet little graphic about your current # of subscribers.

Trackbacks and URL matching

Duncan Mackenzie — Thu, 19 Apr 2007 20:35:00 GMT

I recently noticed a bug in the trackback code on Channel 10, some sites were failing to send us trackbacks, even though we were receiving trackbacks from other sources and those sites were happily trackbacking (I'm just annoying the spell checker today) other blogs.

Turns out the problem appears to be in our RDF block on our blog pages. On a page like this, the RDF block contains the following element:

dc:identifier="http://on10.net/Blogs/tina/exclusive-video-first-look-at-the-xbox-elite/"

This uses the permalink URL that we provide for this page, but of course there are several other urls that work to get to the same resource including

http://on10.net/Blogs/tina/exclusive-video-first-look-at-the-xbox-elite/default.aspx,
http://www.on10.net/Blogs/tina/exclusive-video-first-look-at-the-xbox-elite/ and
http://www.on10.net/Blogs/tina/exclusive-video-first-look-at-the-xbox-elite/default.aspx

When a client, some other blog or blog posting software, goes to auto-discover our trackback URL they try to match the dc:identifier to whatever url they used in their post. So, if they didn't use the one we've chosen for our dc:identifier, then they don't find our trackback info, they don't send the trackback, and we don't get the trackback.

This sucks, but what is the right way to handle it? Add four distinct RDF blocks? Or is there some way to include all four URLs in the one RDF block?

Check out the new Community Bar on on10.net

Duncan Mackenzie — Wed, 29 Nov 2006 08:15:00 GMT

One of my jobs over the past couple of weeks has been to build a javascript based 'toolbar' that could run across the top of on10.net and show folks headlines aggregated from a variety of Microsoft Community sites. So, along with a bunch of help from Adam and Erik, I managed to get it finished and deployed today. You can see it on top of the page on 10

If it overlaps with the logo for you, try forcing a refresh using ctrl+f5 in IE... or just picking reload/refresh in whatever browser you use... the css had to change to accomodate this new content and the old css often sticks in the cache longer than it should.

Categories vs. Tags in Blogs and Blog Editors

Duncan Mackenzie — Wed, 27 Sep 2006 15:48:00 GMT

I've been pondering categories vs. tags in blogs, in my sites (such as http://on10.net) we've completely done away with the concept of categories as distinct entities from tags, which works fine on the site but is causing me a bit of a headache when I look at blog editing and blog editing APIs.

I've enabled the metaweblogapi on my sites so that standard blog editing tools like performancing, windows live writer, etc... can be used by our staff to post entries, and I've used the 'categories' area of that API to represent tags.

This has caused me two issues:

First, there are many more tags on most sites than there would be categories, so editing tools don't always provide the most useful UI for selecting tags.

Second, categories are fairly static, but tags are continually being added... and most blogging software doesn't provide a mechanism for adding to your list of categories.

I'm looking for thoughts, ideas, arguments... anything to help me figure out what the best way is to handle this move from categories to tags while still supporting the standard API mechanisms.

The on10/Channel 9 Dev team have another dev focused video up

Duncan Mackenzie — Wed, 16 Aug 2006 16:38:00 GMT

I even got to be the 'preview' image! Check this out to hear a bit more about our design and implementation of on10.net, and for some more info on our plans around on10.net going forward and for moving Channel 9 on to this new code base.

Windows Live Writer

Duncan Mackenzie — Wed, 16 Aug 2006 15:30:00 GMT

on10.net has a good interview up about the new blogging tool from the Windows Live team. What's cool about this app? For a developer, there is tons to be excited about, including a very easy to use SDK (wrote some plug-in code last night, super easy) and it ships with a bunch of .NET assemblies that I'm about to go spelunking through...

on10.net's tagging gets a little bit more 'connected'

Duncan Mackenzie — Thu, 06 Apr 2006 23:48:00 GMT

Erik extended our tag pages on on10.net a bit, check out the full details here as part of our on10.net news blog.

Found some pre-existing source code to send test pingbacks

Duncan Mackenzie — Wed, 05 Apr 2006 15:56:00 GMT

I wrote my own for Trackbacks, which I used to incorporate trackbacks into http://on10.net, but Geoff provides one on his blog (that doesn't appear to support comments, but does support trackbacks, so I thought I had better blog about it to thank him) that does both.

Silly me, I hadn't built pingback support into on10.net, thinking that Trackbacks would handle most cases... but it turns out that many sites (including wordpress) only send pingbacks... oh well, at least it will be there after our next sprint.

Learn a bit about how on10.net was made

Duncan Mackenzie — Mon, 13 Mar 2006 17:11:00 GMT

Interview with the dev team (including me) is up on Channel 9

Odd Language element in MSN Spaces feeds?

Duncan Mackenzie — Mon, 06 Feb 2006 07:37:00 GMT

Perhaps a little bug left over from when they first launched in Japan, but the MSN Spaces feeds seem to specify a language code for at least one english language feed (Brian's)... but it does return en-US for some others. Hmm... it also seems to specify en-US for some feeds that are quite obviously not... perhaps choosing your language is a per-blog setting that people aren't aware of? I'm not sure if it matters all that much, other than to blog indexers, if the character set and encoding is correct, the lanugage element isn't really important to aggregators or browsers, but since it changes from feed to feed it must be based on some sort of per-blog or per-user setting.

Ah the web, it can change so quickly...

Duncan Mackenzie — Tue, 17 Jan 2006 14:44:00 GMT

Yesterday I posted about the use of the feed icon on Media Center Sandbox and suggested a few changes. Last night, I get an email from Charlie Owen, one of the principals of the sandbox site, and the site is completely updated. Now, I don't know if I caused any change or if the change was just in the works anyway.... but regardless, I like the new look and the icons look great!

Nice to see other folks using the firefox/IE7 feed icon on their sites...

Duncan Mackenzie — Mon, 16 Jan 2006 15:53:00 GMT

Found the Media Center Sandbox today, via Erik Porter, and it is nice to see them using the newly blessed icon. I'd suggest they use a smaller one though, and update the feed icon next to their blogroll to be consistent... but otherwise its great :)

Implicit relationship between search crawler and content providers...

Duncan Mackenzie — Thu, 12 Jan 2006 03:47:00 GMT

A discussion at work today, along with my earlier post about Phil's issues with Google and MSN, had me searching like crazy for this comment:

"I think they're also vulnerable to being locked out of sites because they've missed the implicit bargain for search engines that my site being crawled results in listing for me. Without any public facing services how do I find out what value I am getting from allowing WebFountain to crawl my site ?" --- Matthew Walker's comment on a Searchblog piece about WebFountain

At what point do you block a search crawler? If you don't recognize it? If you don't like the results provided by the search engine it is crawling for?

Term Extraction

Duncan Mackenzie — Tue, 10 Jan 2006 13:51:00 GMT

I was playing around with Yahoo's term extraction service, as a possible path to tagging (this is the same service used behind TagCloud.com, for example), and I created this C# class libary with a sample app written in Windows Forms.

Code is available for download here: Term Extraction (ZIP).

Have you tried my RSS editor? Comments?

Duncan Mackenzie — Thu, 05 Jan 2006 05:50:00 GMT

Ok, so I'm fishing for comments... not compliments at least, but I'm still fishing... I've seen a fair bit of traffic to the install point but not a single email or blog comment about this app... go ahead fire away (and yes, I know that it doesn't do Atom feeds... yet).

Updated my personal site to use the Firefox/IE7 feed icons...

Duncan Mackenzie — Sat, 31 Dec 2005 03:14:00 GMT

As per the blog entry "Feed icons" and using icons pulled from the new feedicons site. Hopefully MSDN will follow suit as well, which would probably be pretty easy... it looks like they are mostly linking every instance of the 'old' RSS icon to the same graphic. I'll suggest it and we'll see what happens.

Added some additional info to each post from my blog, a "bookmark this with Del.icio.us" link and the oh-so-trendy 'tags'

Duncan Mackenzie — Wed, 21 Dec 2005 17:13:00 GMT

Check out a single entry view on my site to see the new additions

Immediate Action Feeds

Duncan Mackenzie — Sat, 17 Dec 2005 08:27:00 GMT

Kent pointed me to this article on xml.com, talking about the benefits of having direct actions available as links in your feeds. This is a good idea, but it is a good idea for any HTML content. If you are going to have a link at the bottom of a MSDN page saying "give us feedback", it would be best to either make sure that link takes the user directly to a form to enter feedback or instead to let them enter their feedback right there on the page. Good stuff all around, but in the context of feeds I have a problem with the way it is implemented.

The suggestion in the article, and it works just fine, is to just add these immediate action links into the body of your feed item. This will seem like a meaningless distinction to some, but to me that is not the best way to go. The links are about the content of the post; they themselves are not the content, so they don't belong in the content.

Having them in the markup of the post means that every feed producer will put their 'actions' wherever they want them, and you can bet it won't be in any consistent fashion. Instead, what I would like to see is a list of 'actions' as a separate part of the feed item. Then the software that is displaying the feed could implement a common method of displaying the list of actions associated with a specific item. In one application this might be a right-click menu option, while in another it could be a sidebar along side the post.... but because the actions are clearly separated from the content it opens up the possibility of displaying them in whatever fashion the application author believes will be most usable.

So, instead of this

<item>
 <title>New Album out by GreenDay</title>
 <description>
 Cool new album hits stores today.... <a href="...">buy it</a>
 </description>
 <link>http://www.musicsite.com/newalbumout.aspx</link>
 <guid isPermaLink="true">http://www.musicsite.com/newalbumout.aspx</guid>
 <pubDate>Tue, 25 Oct 2005 22:54:16 GMT</pubDate>
</item>

We would have this...

<item>
 <title>New Album out by GreenDay</title>
 <description>
 Cool new album hits stores today....
 </description>
 <link>http://www.musicsite.com/newalbumout.aspx</link>
 <ia:actions>
 <ia:action name="Buy It">
 http://www.musicsite.com/buy.aspx?id=greenday
 </ia:action>
 </ia:actions>
 <guid isPermaLink="true">http://www.musicsite.com/newalbumout.aspx</guid>
 <pubDate>Tue, 25 Oct 2005 22:54:16 GMT</pubDate>
</item>

Now it is easy to add additional actions without any change to the content

<ia:actions>
 <ia:action name="Buy It">
 http://www.musicsite.com/buy.aspx?id=greenday
 </ia:action>
 <ia:action name="Email this Post">
 http://www.musicsite.com/email.aspx?id=greenday
 </ia:action>
 </ia:actions>

application/rss+xml vs. text/xml

Duncan Mackenzie — Mon, 12 Dec 2005 16:03:00 GMT

I've been working on some feed support in MSDN's new online platform (a beta of which is running http://msdn2.microsoft.com) and I had to decide what content-type to use when outputting a RSS feed. I knew this was a contentious issue in the past, but I thought it might have been resolved so I did some browsing of specs and discussions and ended up with the following links:

A discussion on Sam Ruby's blog around content-type (the comments are the interesting part)
This post by Dave Winer

I'm sure I could find more, but it appears this was never really resolved... using application/xml seems the most 'proper', but the concern is that some browsers don't know how to handle it ... so the other choice is text/xml (specifically text/xml; charset=utf-8 or else the charset will default to US-ASCII). Hmm... which to choose? Even our own sites have multiple implementations:

The main MSDN feed is application/xml
while the MSDN Magazine's (which is dynamically generated using ASP.NET) is output as text/xml; charset=utf-8

Interestingly enough, I found one feed that used application/rss+xml (Sam's RSS 2.0 feed) which I think is probably not the best choice since that content type was never officially registered, and it was the only feed I hit that IE didn't understand (and therefore tried to just download).

I think I will go with "application/xml" which has the best features in my opinion.

It clearly indicates that this is not just text, so it should avoid issues with proxies messing with the characters,
it leaves the character set data in the xml declaration, avoiding a possible conflict if I specify one in the http headers that is different than what the feed specifies,
it displays correctly in IE and Firefox, and
it is consistent with what we are doing today with the MSDN main feed.

Now, what about those in-page links we have? <link rel="alternate" type="application/rss+xml" title="blah" href="rss.xml" /> .... perhaps they should be just "application/xml" as well?

More on that RSS editor...

Duncan Mackenzie — Thu, 08 Dec 2005 05:28:00 GMT

In the end, MSDN didn't end up needing the RSS editor I wrote, so I've stripped out the features and content specific to their needs and started turning it into a more generic RSS creation tool. If you'd like to see it, as it is, you can run it from a ClickOnce install point here on my site. There are more features coming, and I'm revamping the style and behavior of the category system.... but you can try it out against copies of your own feeds or click File | New Feed to make new ones. Check out the Validate option on the menu, it will send whatever feed you have open to a web service enabled version of the feedvalidator.org system (coded up by Kent Sharkey), allowing you to check the validatity of your RSS without having to publish the file anywhere.

Note: If you have Beta 2 of the .NET Framework installed, the Click Once application will not install or run correctly. In this case, you need to run the VS remove tool (available from here)

Added category elements to my RSS feed, since I wasn't playing Xbox 360

Duncan Mackenzie — Sat, 26 Nov 2005 09:03:00 GMT

About a year or so back I added <category> elements to the MSDN RSS feeds, which seemed like an obvious addition, but it wasn't until recently that I noticed that my own feeds (coming out of my .Text 0.95 based blogging engine) didn't have categories specified in them at all. Not in the main feed or in the per-blog feeds...

This seemed really odd to me, since much of the UI of the .Text posting page, editing page, and even the blog skins themselves is dedicated to the listing and selecting of categories. Obviously, categories are seen as important information about blog entries, so why not include them in the RSS feeds? Oh well, there is little point in wondering about these sorts of things, not when you have the source, so I added category elements to my main and individual feeds. I haven't added them to the per-category feeds yet, or to the ATOM feeds, but I'll get to those in the near future.

If you haven't looked at the .Text source yourself, you might be wondering why adding these elements to one feed wouldn't have added them to all of the feeds, because all RSS feeds are probably running through the same code path. While this is mostly true, they are running through the same ASP.NET handler and through the same feed generation code, the category-based feeds use a different stored procedure to retrieve their entries than the feeds that I have updated, and I had to make a change to the database query to return the list of categories along with each item. What I ended up doing, (and I'm not sure about the performance of this code but it is so highly cached that I'm not particularly worried about it for this use), was using a Function to retrieve the list of categories as a semi-colon deliminated string given a PostID (note that if you host multiple blogs on your .Text instance that this function should take both a BlogID and a PostID... I'll have to update this for the multi-blog case).

 
 CREATE FUNCTION blog_GetCategoryTitles (@PostID int) RETURNS nvarchar(4000)
 BEGIN
 DECLARE @CategoryList nvarchar(4000)
 SELECT @CategoryList = COALESCE
(@CategoryList + ';', '') +
 blog_LinkCategories.Title
 FROM blog_Content
 LEFT OUTER JOIN blog_Links 
 on blog_Links.PostID = blog_Content.ID
 LEFT OUTER JOIN blog_LinkCategories
 on blog_Links.CategoryID = blog_LinkCategories.CategoryID
 WHERE blog_Content.ID=@PostID AND
 blog_Content.BlogID = blog_Links.BlogID AND
 blog_LinkCategories.Title != ''
 RETURN @CategoryList
 END

thanks to Garth's 2001 article from SQLTeam.com for showing me COALESCE being used for this purpose...

and then modifying the queries that retrieved entries to also return blog_GetCategoryTitles(<PostID>), and then modify the RSS writer to output the categories if any were returned.

Interesting note, I figured the omission of category data from the feeds in .Text was a simple error and that it would have been added along the way to Community Server, but I noticed that the feeds on blogs.msdn.com don't appear to have category data either... is category information not considered useful in feeds?

RSS feed authoring for those without blog software or an enjoyment of typing angle brackets

Duncan Mackenzie — Wed, 02 Nov 2005 04:56:00 GMT

Blogs and blogging software seem to be everywhere these days, and RSS has been a top buzzword for quite some time, everyone and their dog wants to take advantage of this new trend and technology. The problem is, it isn't a simple process to create and maintain a valid RSS file. If you aren't willing to run a complete blogging system or if you aren't capable of hand-editing XML, then you don't have a lot of options. For most of the folks that will read this blog entry, you probably don't have this problem, producing RSS 2.0 wouldn't be much of an issue for a developer, but there are times when we want less technical folks to be able to author their own feeds without any assistance. At MSDN we started thinking about this very problem ourselves recently when we decided that, in addition to all the feeds that come out of our content systems, there was a need to create some small feeds that didn't necessarily fit into our larger content systems. Handing off the task of feed creation to notepad or Front Page wasn't an appealing thought and that path would probably result in a lot of xml editing errors and invalid feeds.

This problem happened to line up with a sample I had been thinking of though, so I wrote a quick app using VC# Express 2005 to try and help out; a Feed Writer that allows you to create new RSS 2.0 feeds, edit existing ones, and even import entries from one feed to another. I stuck to a tried and true UI structure, tree along the left side then entry fields on the right:

This app has been developed without the general user in mind, MSDN/TechNet were the targets and because of that there are some fields in this UI that are only relevant to the needs of those groups. For example, the list of attributes you can see on the lower-right is specific to the needs of MSDN and TechNet, who need to markup the feed entries with the appropriate choices. The "Type" and "HeadlineImage" fields are also specific to MSDN feeds, I'm planning to adapt it to work with 'standard' RSS 2.0 items and the category element to make it more general purpose, but for now I thought I'd show you the version I already have running.

In a rather backwards fashion, I'm going to finish up this as a sample and write the article, now that I've finished the actual practical version of the same system... but it will all work out in the end.

Moved my .Text blog from Atom 0.3 to Atom 1.0

Duncan Mackenzie — Wed, 02 Nov 2005 04:11:00 GMT

I only recently noticed that my blog software (.Text 0.95) had atom support built in, so I added a button to my main page, and then I noticed it was Atom 0.3. Since 0.3 was recently deprecated, I updated the Atom generating code to output 1.0 instead and voila;

Much thanks to rakaz's great guide on moving from 0.3 to 1.0, which enabled me to update my code with almost no knowledge of Atom 0.3 or 1.0 :)

Have you heard about SoloSub?

Duncan Mackenzie — Wed, 26 Oct 2005 02:57:00 GMT

I was considering adding one (or more) of the many aggregator-buttons (subscribe to this feed in newsgator, add this feed to your MSN start page, etc.) when I found this, one button that handles all of the most popular aggregators through a single link.

Check out http://solosub.com for more info...

Added Atom support to the FlairMaker... my version at least

Duncan Mackenzie — Sun, 23 Oct 2005 05:19:00 GMT

Nick, from "The Coding Monkey" mentioned that he would like to see Atom support in the FlairMaker, so ... having no real reason not to, and having an extra 15 minutes after the kids went to sleep before I started cleaning up the house, I added it...

Of course, I've just added this to my forked version of Kent's original code... so any feature I add that he likes will need to manually added into his version... bad software development practice, but hey we're just playing around here so you should be expecting too much.

Update: I also modified the code to pull the blog's title from the feed (Atom or RSS) and stick it at the bottom. Not that useful of a feature for the Coding 4 Fun version, since it is always pointing at the same feed... but useful for mine if people want to use it for their own feeds

After some discussions with Sam Ruby and others on the FeedValidator mailing list, the MSDN RSS feed validates as is...

Duncan Mackenzie — Wed, 28 Sep 2005 02:39:00 GMT

In an earlier post, I discussed the fact that the MSDN feeds were failing to validate due to a MIME type that included parameters (charset in this case, like 'text/html ;charset=utf-8'), but I also posted a query about this issue into the listserv for FeedValidator.org. Sam mentioned it on his blog, and then went ahead and updated the validator to recognize a MIME type with parameter as valid.

In the meantime, I updated the MSDN generator to strip out the parameters :), but I still think they are technically valid so I'm glad the feed validates as it is today (with params) and as it will exist in the near future with the MIME types stripped down to just type/subtype.

A bug in my RSS generator, but is it really invalid?

Duncan Mackenzie — Tue, 27 Sep 2005 04:36:00 GMT

The RSS generator for MSDN, creator of this feed, and many more ... has a small problem. Way upstream, when various people inside the company enter information about an upcoming headline, they have the ability to specify a URL to a download. The intent was for this to be a URL to an actual downloadable file, so when I generate an RSS item from that headline entry, I take that URL and turn it into an enclosure entry in the RSS file.

    <item>


      <title>Read about Atlas - Ajax for ASP.NET</title>


      <description>ASP.NET "Atlas" is a package of 

new Web development technologies that integrates an extensive set of 

client script libraries with the rich, server-based development 

platform of ASP.NET 2.0. </description>


      <link>http://msdn.microsoft.com/asp.net/future/</link>


      <dc:creator>Microsoft Corporation</dc:creator>


      <category domain="msdndomain:ContentType">Link</category>


      <category domain="msdndomain:Audience">Developers</category>


      <category domain="msdndomain:Hardware">CPU</category>


      <category domain="msdndomain:Operating Systems">Windows</category>


      <category domain="msdndomain:Subject">Web development</category>


      <msdn:headlineImage />


      <msdn:headlineIcon>http://msdn.microsoft.com/msdn-
online/shared/graphics/icons/offsite.gif</msdn:headlineIcon>


      <msdn:contentType>Link</msdn:contentType>


      <msdn:simpleDate>Sep 19</msdn:simpleDate>


      <enclosure url="

http://go.microsoft.com/fwlink/?LinkId=52384" 
length="17437" 
type="text/html; charset=utf-8" />


      <guid isPermaLink="false">Titan_2519</guid>


      <pubDate>Mon, 19 Sep 2005 18:20:40 GMT</pubDate>


    </item>

This generally works fine, I make a HEAD request with that URL which gives me back the MIME type and the Content Length, both of which are needed for the enclosure element in the RSS item. Sometimes though, people put in a URL to the download's landing page, not the download itself. There are good reasons for this, as the download page often contains useful information and/or multiple localized versions of the download, but it was not what I expected. In this case, I put the enclosure in with the MIME type I get back from that URL, which ends up being 'text/html' and with a byte size that reflects the size of the landing page.

This wasn't really what I wanted to happen, so I need to figure out a solution at my end, but what I noticed today and what has me a little puzzled is that at least two different validators (here and here) report these types of entries as validation errors. The error they specify is that text/html is not a valid MIME type.... but, according to the RFC(s) (see 4.1.2 of this RFC) and other sources, it most certainly is a valid type. So, is there a hidden rule in RSS that enclosures have to fall within some special subset of MIME types, or are both of these validators broken? Sure, in this case it wasn't really what I wanted, but what if I really did have a text/html document for you to download?

Long overdue, but here is the source for the "blog copier"

Duncan Mackenzie — Sat, 02 Apr 2005 06:32:00 GMT

In my first post on this site, I mentioned an application that I wrote to copy blog posts from one .Text server to another. Overall, I didn't think the application had any real use after the web service I used was removed from the weblogs.asp.net servers, but I'm posting it anyway for anyone who might find it useful.

Source here: http://www.duncanmackenzie.net/Samples/#blogcopy

More on "pulling" MSDN content into my site...

Duncan Mackenzie — Mon, 21 Mar 2005 18:48:00 GMT

In my last post, I was talking about pulling my articles from MSDN into the chrome of my site. This type of system could be created using a frameset, but frames are evil, so that isn't the approach I took. Instead, knowing a bit about the files on MSDN's web servers, I took advantage of a special xml file that exists for most of our articles. This file is created as part of our publishing process and exists so that we can pull articles into the chrome of our developer centers (like this). It isn't a straight xhtml file, but it is almost identical to the html content of the article itself. Knowing that this file exists, my pull code just munges the original (MSDN) URL of the requested article to figure out the underlying xml file name, then loads up that xml. Once I have the xml content, I do a bit of work to the elements, to make all the relative links correctly point back to MSDN (for the images, related articles, links into the SDK, etc...) and then output html into a placeholder on my own page.

Given a URL like this, you can remove the pull syntax (used by our developer centers) to come up with the library URL of this article, /library/en-us/dncodefun/html/code4fun12102003.asp, then apply a complex transform to produce the likely URL of the 'behind-the-scenes' XML file: /library/en-us/dncodefun/html/code4fun12102003.xml.

It is possible that this XML file doesn't exist, so it is important to handle that possibility in your code. In my case, if I can't find the XML I just redirect to the original MSDN url. If the original URL doesn't appear to be well formed, I just give up completely and redirect to the home page of my own site.

more details and code to follow....

[Listening to: Last Chance - Jet - Get Born (01:52)]

Playing around with pulling my articles into my own site's chrome...

Duncan Mackenzie — Sat, 19 Mar 2005 06:36:00 GMT

Check out my articles page, which is built from an RSS file by the way, and click on any of the "Coding 4 Fun" articles. You'll end up still in my site, but viewing the full MSDN article. Mostly I'm just playing around, but it seemed 'neat' enough to mention :)

By the way, the jury is still out on whether or not I've opened myself to various new security risks through this code, so if you want to try and hack my site using this new "pull" concept, go for it... I'd really appreciate it if you'd let me know if you find anything though, in the interest of science that is :)

Just pulled down the CS 1.0 source... now to start working on a migration plan

Duncan Mackenzie — Sun, 27 Feb 2005 03:38:00 GMT

Currently I'm running .Text .94 + a bunch of private modifications and some version of CS:Forums + a bunch of private modifications... so now that CS 1.0's source is out, I want to incorporate my mods into the combined package and then migrate my data and get my site up and running on the new stuff. As cool as that is, and while I'm sure it has excellent tools/scripts for upgrading, I'm not looking forward to it. It was hard enough to get my site set up now, with blogs.duncanmackenzie.net running as its own app domain and then trying to make the Forums code work when the application started at the root not at /Forums. Anyway, suffice it to say I don't expect to upgrade right away, but maybe I'll have a copy of the site upgraded and sitting on my dev box within the next week....

Anyone else gone through the upgrade/migration and have any thoughts or comments?

Halo 2 Stats via RSS... an interesting idea that I think could be improved

Duncan Mackenzie — Tue, 23 Nov 2004 04:31:00 GMT

I like that Bungie is exposing your stats as RSS (see mine here), but it would be cooler if they had added some additional elements and attributes to the feed to describe the games in a more usable format, rather than just in HTML... instead of;

				
						<item>
						

				 <title>Rumble Pit: Oddball on Ascension</title>
 <link>http://www.bungie.net/stats/gamestats.aspx?gameid=8132172&player=Festive Turkey</link>
 <pubDate>Fri, 19 Nov 2004 07:45:07 GMT</pubDate>
 <guid>http://www.bungie.net/stats/gamestats.aspx?gameid=8132172&player=Festive Turkey</guid>
 <description>
 Game played at Fri, 19 Nov 2004 07:45:07 GMT<br/><br/>Playlist: Rumble Pit<br/>Oddball on Ascension<br/><br/> 
 <b>Gamertag (Team): Score, Kills, Deaths, Assists</b> 
 <br/>l obvious l (1): 59, 14, 9, 5<br/>ScottyAK (0): 52, 16, 10, 6<br/>
 BK Assassin (2): 29, 6, 9, 1<br/>MAXODeeZ (3): 18, 7, 12, 4<br/>
 BattlingWheel (4): 15, 10, 13, 5<br/>Festive Turkey (5): 6, 5, 11, 5<br/>
 snackasaurus (6): 0, 6, 7, 6<br/>LionWrath716 (7): 0, 1, 0, 0<br/> 
 </description>
</item>

they could add a namespace reference for Halo or Bungie.... and end up with...

<item>
 <title>Rumble Pit: Oddball on Ascension</title>
 <link>http://www.bungie.net/stats/gamestats.aspx?gameid=8132172&player=Festive Turkey</link>
 <pubDate>Fri, 19 Nov 2004 07:45:07 GMT</pubDate>
 <guid>http://www.bungie.net/stats/gamestats.aspx?gameid=8132172&player=Festive Turkey</guid>
 <description>
 Game played at Fri, 19 Nov 2004 07:45:07 GMT<br/><br/>Playlist: Rumble Pit<br/>Oddball on Ascension<br/><br/> 
 <b>Gamertag (Team): Score, Kills, Deaths, Assists</b> 
 <br/>l obvious l (1): 59, 14, 9, 5<br/>ScottyAK (0): 52, 16, 10, 6<br/>
 BK Assassin (2): 29, 6, 9, 1<br/>MAXODeeZ (3): 18, 7, 12, 4<br/>
 BattlingWheel (4): 15, 10, 13, 5<br/>Festive Turkey (5): 6, 5, 11, 5<br/>
 snackasaurus (6): 0, 6, 7, 6<br/>LionWrath716 (7): 0, 1, 0, 0<br/> 
 </description>
 <bungie:game>
    <bungie:datePlayed>Fri, 19 Nov 2004 07:45:07 GMT</bungie:datePlayed>
    <bungie:playlist>Playlist: Rumble Pit</bungieplaylist>
    <bungie:variant>Oddball</bungie:variant>
    <bungie:map>Ascension</bungie:map>
    <bungie:players>
       <bungie:player>
          <bungie:gamertag>Festive Turkey</bungie:gamertag>
          <bungie:score>6</bungie:score>
          <bungie:kills>5</bungie:kills>
          <bungie:deaths>11</bungie:deaths>
          <bungie:assists>5</bungie:assists>
       </bungie:player>
    </bungie:players>
 </bungie:game>
</item>

still valid RSS 2.0, still works in aggregators, but suddenly useful to anyone wanting to consume this information for more than just directly viewing it...

Added an RSS feed to the site for my articles...

Duncan Mackenzie — Mon, 18 Oct 2004 21:16:00 GMT

I guess I could have just added a category to my blog, but this works... http://www.duncanmackenzie.net/articles/rss.xml

New MSDN RSS feeds are live...

Duncan Mackenzie — Wed, 06 Oct 2004 04:56:00 GMT

In an earlier post, I mentioned that I had written a new system for producing MSDN's RSS feeds, and those feeds are now live at the same URLs as the previous versions. You might get some duplicates in your aggregators, (since these are new feeds but with some overlap of items from the old feeds) but that should be temporary. Anyway, check them out (here is the one for VB) and let me know what you think.

Some people have asked why I haven't gone into a lot of detail about how we generate these feeds. Basically, I haven't talked about it because it isn't all that difficult to write out a RSS file... the trick is usually how you obtain the data that you wish to write out and that part of the system isn't interesting (IMHO) to the general public as it is just SQL queries against our database full of content... the process that generates these feeds runs daily, and then static xml is posted to the site... all in all, not that exciting :)

I've been working on a new RSS generation system for MSDN for the past little while

Duncan Mackenzie — Mon, 27 Sep 2004 04:19:00 GMT

... and it is almost ready to ship....

The new system will result in a few changes to the MSDN feeds, the most notable of which is that our feeds will no longer reflect a certain time span (they currently contain all of the appropiately attributed items from the past 30 days), but will instead contain a certain # of items. They also have a bit more data in them, including the author of the article (in the dc:creator element) and a collection of category elements containing all of the attribution that we have applied to that article in our back end system.

If you check out a sample feed, available here, you'll see the changes described above along with one other new 'feature'. We will have a stylesheet added to the top of the feeds, which is primarily targetted at folks who do not already use RSS feeds. When people in the past have seen our RSS buttons on the various MSDN sites, clicking on that button showed them the raw XML... with no real context or explanation of what they were being shown. This new format, using the style sheet, should help improve the experience for those customers.

Please let me know what you think about these changes (feel free to post your thoughts in the feedback section of this post), they aren't released yet so this is the time when I need to hear any and all feedback you have!!

Update: Luc Cluitmans noticed that the feed was not displaying correctly in Firefox... but I believe I've fixed that problem now, please let me know if you still notice problems with the display. I get only text in Opera, but I think that is due to the fact that XSLT is not supported by Opera... let me know if I'm wrong on that one...