Search Engine Optimization Tweaks over the long weekend

My team builds a fair number of community sites including Channel 8 (for Students), TechNet Edge (for IT Pros), Channel 10 (for enthusiasts, power users and gamers), Mix Online (for web developers and designers) and the original site… Channel 9 (aimed mostly at developers) … and we’ve recently starting putting out sites on a new code base. One of the changes in that new code base was a move to an AJAX style interface for viewing lists of posts on the page. We like the way this works for paging through lists of entries, comments, etc… but we have known from the beginning that it was going to cause us some trouble in the world of search engines and other crawlers. Without JavaScript, there was very little being output onto the page, and what was there was mostly navigational chrome. Taking a look at Google’s cache of TechNet Edge from a few days ago gives this:

not much to see without script

Checking how your page appears in the cache of Google or Live is one way to check how you appear to crawlers, but it doesn’t work great when you are making changes or running in development. One handy way is to check your site using Lynx, like Joshua mentions in this post on Mix Online.

The content on the site was ending up in the index of search engines anyway, through the virtue of RSS feeds and incoming links… but the value of your site to crawlers is going to be much lower than it should be if they don’t see any content when they visit. As I said earlier… we always knew this would be a problem, but I guess we just didn’t get around to fixing it before pushing out a full three sites using AJAX based paging. Last week I had a meeting with a SEO consultant and they pointed out the exact issue I’ve been describing. Well… given a long weekend… and no interest in working on my actual planned tasks… I decided to implement two features to help how our sites appear to crawlers.

First, I added some code that swaps out our fancy Ajax entry list with a simple ASP.NET repeater if the browser doesn’t appear to be one that is supported by Microsoft Atlas, making our site usable to other browsers (Atlas supports the bulk of users, but not all) and also making our content visible to a crawler. So far, I only output the first page of any given entry list, but that makes the results go from blank to this:

current cached version from Live.com

Next, I added an XML sitemap, following the specs from sitemaps.org, by outputting a sitemap index at http://<site>/sitemapindex.ashx and then outputting a series of sitemaps (by page #) from http://<site>/sitemap.ashx?page=<number> (see Mix’s sitemap index, and sitemap as an example). Finally, I put a link to the sitemap index into the robots.txt file for each site.

Between the two, I’m hoping our content will get indexed better by a variety of search engines, resulting in more people finding us when searching for relevant topics. These changes also help to make us a little bit more usable to some users, but that is another area where we need to do a lot more work. If these changes improve our accessibility that’s great, but I’d hate to even suggest that they get us anywhere near our goals in that area.

Picked up Mass Effect today… built XML site maps instead of playing it :(

I’m very excited about it… really! I just got distracted on http://sitemaps.org and ended up building a site map for this site… real brute force, just grabbed all the blog entries and tags and output the appropriate absolute paths into some objects that I then wrote out into an XML document. Same method wouldn’t work on Channel 9 though, it has too many entries, so I’ll need to move to a site map index with multiple sub site maps… now I just have to figure out how to divide the entries up (by tag, by type of content, by date?).

Ice Cube talking about Silverlight…

and the Group Manager of Platform Evangelism at Adobe feels the need to comment about it.

Let me summarize. Ice Cube is involved in an Internet video project and has chosen to use Silverlight, and you can watch an interview with him about it in Silverlight. (on a side note, I decided not to include the video into my post, even though the instructions to do so are right there in the original blog entry, because it is set to auto-start when someone visits the page and I think that is just downright rude 🙂 )

The post is interesting, the video is interesting… but what is really interesting is the fun going on in the comments on that blog entry, including comments from Mike Downey of Adobe and Scott Barnes of Microsoft… fun stuff. The old ‘if you have nothing nice to say…’ rule probably should have been applied for both of them.

 

found via Mix Online

We added a Silverlight player to Channel 8 today

We used a modified version of one of the templates that ships with Expression Media Encoder as our player… and channel 8 has become the first of our sites (which include Channel 9, Channel 10, VisitMix.com and more) to have a Silverlight video player on them. And, below here, you can see that player embedded into this blog post…

Note that I had to remove the ‘scrubbing bar’ due to some odd bug around releasing the mouse after scrubbing was completed… but we’ll figure that out and get it back in as soon as we can. Check out the video above back in its home on C9…. check out the video from above on on Channel 8, and build your own player using those templates!

Added Silverlight Streaming support to Channel 9’s forums

Continuing on in the same pattern as my last post, adding Soapbox support to C9’s forums, Adam Kinney bugged me a bit and I went ahead and added support for Silverlight Streaming as well.

SilverlightStreaming

It will be interesting to see if (or more accurately, how long before) someone turns this to malicious purposes.

Added Support for Soapbox Videos on Channel 9

When I started on the C9 dev team, one of the first things I did was to remove the ability for users to post embed, object and script tags. We had a couple of people abusing this feature and I wanted to lock it right down. The downside to this move was that it prevented some forms of user interaction/posts that we didn’t really mind and in some cases would even like to encourage.

As a small step towards a more community-driven future, I added support to C9 today to allow you to embed Soapbox videos in any forum post. Simple use the following markup in your post:

[soapbox video=”(soapbox video id)”]

image

and it will turn it into a full blown embed tag with a link to the source material underneath. The Soapbox video id is simply the GUID that appears on the permalink url (http://soapbox.msn.com/video.aspx?vid=5699dafe-d864-4d1f-976c-d5f4d9ed78db for example) for a video and also shows up in its embed code if you take a look at that as well.

The Channel 9/Channel 10 team is hiring

Hey folks, I work as the dev lead for Channel 9, Channel 10, Mix Online and other sites. We build community sites for Microsoft using video, audio and text blogging and we build them using the latest .NET technologies. If you are into web development, if you have a passion for design and user experience, and if you really rock at AJAX, Javascript, CSS, C# and ASP.NET …. then this is the job for you 🙂

The Developer and Platform Evangelism Team is hiring a software design engineer to join the team that builds and extends Channel 9, Channel 10 and more. Our sites build a two way interaction between our customers and Microsoft using video as the primary communication tool. In the past several years we have built several additional sites and our network has grown in both traffic and content, and as a result we are looking for an additional Software Design Engineer to join the team. We are looking for someone who lives and breathes HTML, CSS, Javascript, AJAX and ASP.NET. The ideal candidate will have skills in all those technologies combined with a passion for design and user experience.

Do you like to ship, takes chances and effect change? Do you want to build cool sites that use Silverlight and Atlas to deliver a great user experience? Then come join our group where you’ll find a small dynamic team of folks and you’ll swear you are part of startup.

Experience in HTML/CSS, ASP.NET and C# is required and experience with AJAX, SQL, Scrum and Test-Driven Development is highly desired. Experience with Silverlight and/or WPF is not required but it would be great to have.

If you are interested and you meet the requirements, go to the official job listing on the Microsoft careers site and submit your resume!

Trackbacks and URL matching

I recently noticed a bug in the trackback code on Channel 10, some sites were failing to send us trackbacks, even though we were receiving trackbacks from other sources and those sites were happily trackbacking (I’m just annoying the spell checker today) other blogs.

Turns out the problem appears to be in our RDF block on our blog pages. On a page like this, the RDF block contains the following element:

dc:identifier="http://on10.net/Blogs/tina/exclusive-video-first-look-at-the-xbox-elite/"

This uses the permalink URL that we provide for this page, but of course there are several other urls that work to get to the same resource including

  • http://on10.net/Blogs/tina/exclusive-video-first-look-at-the-xbox-elite/default.aspx,
  • http://www.on10.net/Blogs/tina/exclusive-video-first-look-at-the-xbox-elite/ and
  • http://www.on10.net/Blogs/tina/exclusive-video-first-look-at-the-xbox-elite/default.aspx

When a client, some other blog or blog posting software, goes to auto-discover our trackback URL they try to match the dc:identifier to whatever url they used in their post. So, if they didn’t use the one we’ve chosen for our dc:identifier, then they don’t find our trackback info, they don’t send the trackback, and we don’t get the trackback.

This sucks, but what is the right way to handle it? Add four distinct RDF blocks? Or is there some way to include all four URLs in the one RDF block?

Rory gives us the inside scoop on the Microsoft Research TechFest

Microsoft Research TechFest – Intro, DynaVis, and FastDash

… In this, the first video in what will be a series covering this event, we get a nice introduction to TechFest, some info about Microsoft Research (MSR), and two projects: DynaVis and FastDash.

When you watch this video, hopefully you get an idea of why I’m so excited right now. Once I have the whole series online, you’ll be able to see the amazing work being done in MSR – and this only represents a few projects. There are many others. …

Every year at Microsoft there is a little internal-only conference where Microsoft Research (MSR) shows off their projects. It is very cool but very private (badges checked at the door, etc…). Well, this year it is a bit different… For MSR TechFest 2007 they’ve allowed folks to come in and interview the researchers, take pictures/video, whatever… crazy stuff and you get to see the results first on Channel 9.  Rory tours us around, gives us an intro to the whole event and even covers a couple of projects… and this is just the first of the series!