SharePoint and Office 2010 to Launch on May 12th

On Friday, Arpan Shah announced the official debut date for Microsoft Office 2010 and, of course SharePoint 2010, on the Microsoft SharePoint Blog. In the same post, he mentioned that the RTM (Release to Manufacturing) will come a few weeks earlier, some time in April.

There are a lot of changes coming in the new versions, so there is also lots of planning to do. I know many of you are planning to move forward aggressively, while many of you will also be on older versions of SharePoint long into the future. Whichever path you choose, it might be helpful to keep the following in mind:

• Your current stuff will still work, even once the new software comes out. You don't "need" to upgrade immediately.
• SharePoint Server 2010 requires Windows Server 2008. It also requires that your entire stack, including both Windows Server and SQL Server, be 64-bit.
• Although you will always get the best results when keeping both the Office client and SharePoint versions in sync, you will still get reasonable functionality with staged upgrades. (Look for information about just how the different version combinations interact soon.)
• One exception to the previous statement is SharePoint Designer. SharePoint Designer 2007 will not work for SharePoint 2010 sites. Conversely, SharePoint Designer 2010 will not work with anything except SharePoint 2010 sites.
• On the Office client side, even if you are using 64-bit Windows, you can still use the 32-bit Office. This is critical, because you cannot mix and match 32 and 64 bit versions of Office on the same system. Naturally, you can't use 64-bit Office on 32-bit Windows in any case.
• No matter what version of SharePoint you are on, a failure to plan is a plan to fail. Think about how you want to use SharePoint in your company before you deploy it.

This is going to be an exciting Spring in the SharePoint world, and I can't wait to help you make sense of it!

Watch this Space!

My Session

Nevertheless, there were 20 sessions, in four tracks. My session was the first of the day in the "No-code solutions" track. I gave an Introduction to SharePoint Designer 2010 (click the link to download a PDF of the presentation). I touched on many of the key features, but gave special emphasis to how you could easily configure SharePoint to disable the "dangerous" functions (modifying master pages, and editing or un-ghosting pages). This, along with SPD's new emphasis on business process integration, makes it a great tool to give to your business analysts and information workers, without needing to worry about them "messing up" your corporate branding or making their sites unusable.

Collaboration and Camaraderie

Of course, the big benefit of coming to a smaller conference is getting actual face-to-face time with the presenters, without the pressure of competing with thousands of others who want the same thing.

And let's not forget drawing for door prizes!

In the end, knowledge was shared, connections were formed, and a good time was had by all.

I'm really looking forward to the next one!

P.S. In case you missed it earlier, here is a link to a PDF of my presentation.

The Answer may be SharePoint, but Don't Forget the Questions!

One of the biggest reasons some SharePoint deployments fail is because they are "SharePoint Deployments".

People hear the buzz, and want to jump on the SharePoint bandwagon. They buy servers, attend training, install the software. Big bux are spent customizing and branding a SharePoint home page. Maybe there's a big roll-out promotion. Everybody says "Look! We've rolled out SharePoint!". And then...

Crickets.

All Dressed Up, and Nowhere to Go

But, you ask, what about all of those stories you hear about "uncontrolled growth"? People clamoring to get their own SharePoint sites? That's all true as well, but you need to consider why that is happening. In those cases, the people have a goal, and find that SharePoint is a great tool for making that goal a reality. The goal isn't to have a SharePoint site, per se. Rather the goal might be "easier document and calendar sharing", and SharePoint is used to attain it.

Simple Pleasures

Often those implementing SharePoint forget the KISS principle (Keep it Super Simple). SharePoint has a lot of great features and functions right out of the box. It is very tempting to try implementing all of them at once on the same site, sometimes even the same page. It is almost like when "desktop publishing" was made possible by Postscript laser printers. People discovered how easy it was to have a dozen fonts on a single page, and so that's what they did.

Similarly, in the early days of the web, as new features were added to web browsers, they started showing up everywhere on sites. (Does anyone remember the <Blink> tag?) And don't even get me stated on some of the early Flash-based sites. It got to the point where IBM was even poking fun at the designs in their commercials. "It's a Flaming Logo!" Why? Because we can!

The fact is, like the flaming logo, the fanciest features SharePoint has are worthless unless they are used in the service of some actual user need. For example: Assuming it is reasonably well implemented and up to date, the most used feature on any intranet is almost guaranteed to be the company phonebook or employee directory. Probably by an order of magnitude above any other function. It isn't fancy, but it is something people actually need on a regular basis.

As it turns out, SharePoint, with its personal profiles and My Sites, makes a great employee directory. :)

Form Follows Function

Of course, if all you needed was an employee directory, SharePoint would be overkill in the extreme. But put that directory in the context of a company intranet, with news and knowledge bases, collaboration and search. And here's a radical idea - Ask your users what they need first, and implement that! Maybe throw in a few things that are just for fun, like classified ads, or pictures from the company picnic. Suddenly you have a "destination" that will draw in your users and enhance the sense of community in your organization.

These are all things that could be (and often have been) done individually without SharePoint. But SharePoint gives you the tools you need to build and maintain these "applications" easily, quickly, and consistently - usually without custom code.

Now you can add your branding, and promote "Our-Net 2.0". Sure, it is a site based on SharePoint, but now you have put the horse before the cart, and given your users the tools they really need. It doesn't matter to them what the name of the technology behind the scenes is. All they care about is that you have created something that can help them do their jobs better.

Let SharePoint play Clark Kent, so you can look like the super hero.

On the Road Again...

Well, 2010 has just barely gotten started, and SharePoint Saturday's are already in full swing. I'm pleased to announce that I've been selected to present at the SharePoint Saturday in Indianapolis, Indiana. This takes place on January 30th, 2010, at the Gene B. Glick Junior Achievement Center. Click on the link or logo above for all the details, including registration, a list of the other presenters, as well as the Twitter feed of #SPSIndy commentary.

A SharePoint Saturday is a free to attend event that serves as a mini SharePoint conference. You get some of the same world-class speakers and content found at the big events like Tech-Ed. If you're in the Indianapolis area, and want to know more about SharePoint, this is not something you'll want to miss.

The Obligatory Year-end Report

It is now the last week of December, and you know what that means. Lying in wait among the crumpled wrapping paper, danced-out sugarplums, and pine needles (not to mention the feathers and other "presents" from all those birds your true love gave to you) you'll find year-end wrap-ups from every corner imaginable.

This is mine. :) I'll get to the more general stuff in a moment, but there was one personal SharePoint-related thing that stood out for me in 2009, and that happened the very first day.

I Became a Published Author

While I (along with Asif and Bryan) did all of the work in 2008, my book Professional Microsoft Office SharePoint Designer 2007 was officially released by Wrox Press on January 1st of 2009 (December 31st, 2008 in some markets). I'm honored by the very positive reviews, and want to thank everyone who has purchased it.

Ironically, its importance is going to continue on into 2010, and possibly beyond. While SharePoint 2010 is top-of-mind for many people right now, the fact is that SharePoint 2007 is going to be around for a long time to come. But, SharePoint Designer 2010 cannot work with SharePoint 2007 sites, so SPD 2007 will still be needed if you want to customize older SharePoint sites. Since SPD is now a free download, there isn't much in the way of printed documentation, other than (you guessed it) third party books - like mine.

The Year of the "Community Conference"

One amazing thing that stood out about 2009 was the popularity of independent SharePoint-related conferences and seminars. I personally presented at two - the "Best Practices SharePoint Conference" in San Diego, and the "SharePoint Technology Conference" in Boston.

But the big trend was the emergence of the "SharePoint Saturday" mini-conferences. These are one-day, free to attend conferences, that are held all over the world. Here you will find breakout sessions by the very same experts that present at the larger venues, like Tech-Ed and the official Microsoft SharePoint Conference. Check out the SPS site, and plan to attend one of these events near you!

And, of course, no mention of the "Community Conference" would be complete without mentioning the "Conference Community" on Twitter. This is a bit less formal, but essentially you will find a play-by-play for almost every conference being happily tweeted by the attendees with hash tags like #SPC09, or #SPSINDY.

SharePoint 2010

Of course, the big news was the announcement of Office and SharePoint 2010, and the availability of the public beta. The official release is currently set for the "first half" of calendar 2010. As many articles have pointed out already, much has changed. There will probably still be some significant changes between now and the final release, though what they might be, nobody can say.

Other Significant Events

Some of the other SharePoint things that happened to me in 2009

• I became Michael Gannotti's very first "Backseat Driver"
• I was once again awarded as a Microsoft MVP for SharePoint Server
• I autographed and gave away almost 500 copies of my book while working the Microsoft Technical Learning Center booth at Tech-Ed in Los Angeles

SharePoint, the Target

No post about SharePoint in 2009 would be complete without some mention of another buzzphrase that started appearing in 2009 - "SharePoint Killer". Almost every new application that had the slightest thing to do with collaboration or content management seemed to earn headlines of "Is X the Next SharePoint?" or "Y will make SharePoint Obsolete". From Google Sites to Google Wave. From Drupal, to Alfresco, to the classic DotNetNuke. Yet while each may have one area where it shines, none of them really has the versatility or power to match SharePoint, assuming they are even truly comparable.

Blog Highlights

For those of you new to my blog, here are some of the articles I wrote this year that you might find particularly interesting:

• Press F1 - SharePoint Help is on the Way
• Selecting Editors in SharePoint Designer
• Something Old In Something New
• Binary Free SharePoint Twitter Search Web Part
• A Hidden Gem - the Preview Pane View in SharePoint
• Wiki-in-the-Box - Is SharePoint Wiki Really that Bad?
• On Babies, Bathwater, and SharePoint Designer
• Discovering the Setup User Account - A SharePoint "Whodunit?"
• SharePoint on a Shoestring
• Office 2010 - A Quick Look
• Advanced Calculations in SharePoint Designer Workflow (Part 1)
• Calculated Columns and SPD Workflow Part 2
• Indexing SharePoint List Columns
• Knowing Your Limitations
• A Free SharePoint Org Chart Web Part
• SharePoint Virtualization

Looking Ahead

With the planned official release of SharePoint 2010, 2010 the year looks like it will be just as exciting as 2009. One thing that is very clear is that Microsoft is putting a lot more effort into the supporting infrastructure for SharePoint. From native support in Visual Studio, to online documentation, to partner training.

While people may have been shocked by SharePoint's meteoric rise, nobody is going to be surprised by its continuing momentum.

"You Must Un-learn what You Have Learned!"

The public beta of SharePoint 2010 has been out for a few weeks now. Many people are discovering and blogging about some of the great new features you're going to find. Yet there have also been some significant changes to existing features. These are things you may have been using every day in SharePoint 2007 and WSS 3.0, but which in SharePoint 2010 have moved or changed in ways could cause confusion to experienced users.

In this article I'm going to focus on changes the typical end-user would see. In future articles I'll talk about changes for site owners and administrators.

If it Ain't Broke…

SharePoint 2007 took a lot of heat for having certain "quirks" in its user interface design. For 2010, much as they did for the Office clients in 2007, Microsoft put a lot of R&D into what it would take to make SharePoint easier for typical users. This resulted in a lot of changes.

Human beings are creatures of habit. With certain notable exceptions, we don't much like change. Despite having worked through a non-intuitive learning curve, or sometimes because of it, we would rather keep doing things the way we are used to than learn new ways - even if those ways are better.

...Fix it Anyway

Only time will tell if the changes Microsoft made truly are for the better, but they've definitely been made. Let's start by looking at the basic team site page in SharePoint 2010 side by side with its SharePoint 2007 equivalent:

At first glance, they're pretty similar. The WSS logo has been replaced with a "real" picture, but there's still a banner, title area, quick launch, and content space. But look a little closer. The Site Actions menu has moved. No big deal there - lots of custom master pages move that around. But, the new placement is comparable to the Backstage/File menu in the new Office 2010 client applications, thus making it a "natural" place for users to look for "application"(site)-wide functions. This analogy becomes even more obvious when some of the other tabbed interface options start showing up. (You'll see that later in the article.)

Where's MySite?

Another subtle change is the "personal" section of the banner. In SharePoint 2007, you had separate entries for User ID, links, and a direct link to your "My" site.

All of these options are now accessed through the menu under your name. There is also no reference to My "Site", rather it simply calls it your "Profile".

I think the hope here, is that by de-emphasizing the "independent site" aspect of the profile and personal storage, while actually expanding its function (the new profile features could fill up several articles on their own), resistance to deployment in certain enterprises would be reduced.

Bread-Crumbling Navigation

Getting around from site to site, and from place to place within a site, has received a LOT of attention in SharePoint 2010. In many cases, this has meant "reimagining" the concept of a breadcrumb.

In the case of 2007 site navigation, a breadcrumb stretched across the top of the page content area:

For large site hierarchies, this could become unwieldy as it stretched across the page. For 2010, Microsoft replaced it with a folder icon in the tab banner, which produces an indented hierarchical view of your current location:

Going the other direction, in SharePoint 2007 lists and libraries selecting a view was accomplished by selecting it from a drop-down list on the toolbar.

In SharePoint 2010, there is no list toolbar. While you can drill into the ribbon and find the view settings, then select your view, that's a lot of clicking. Fortunately, Microsoft has turned the title panel into an in-site breadcrumb. When looking at a list or library, the last item in that breadcrumb is the name of the current view. If you look carefully, you'll notice that there is a "down triangle" arrow. That's your hint that this element is actually a dropdown menu, where you'll find all of your view selecting goodness.

Tied up with a Ribbon

Of course, the rest of the stuff that used to live on a list or library's toolbar:

has been moved into the Library tab of the new ribbon interface:

By the same token, individual items that lived in an individual item's dropdown:

have been moved into the Documents (or other appropriate item's) tab:

Note: In this case, the individual item dropdown is still there as well.

Summary

Some folks say, "The more things change, the more they stay the same". There have been a lot of changes in SharePoint 2010. While there are some things that have stayed the same, they are actually in the minority. In this article, I have gone over some of the many changes to "carry over" functionality you will find when moving from SharePoint 2007 to SharePoint 2010. There are many more than I could hope to address in a single posting. I hope, however, that this article has given you some ideas of where to look if you can't find your favorite function where it used to be.

The Fix is In, Thanks to Tom Resing

I have great news, thanks to fellow SharePointer Tom Resing. In my previous post I mentioned the problems the Community Kit for SharePoint:Enhanced Blog Edition has with the new link tracking parameters FeedBurner just started adding to their links.

In that post, I talked about the trials and tribulations of trying to get CKS:EBE working by installing an updated version. It turns out there was another approach to the problem. Although Google made the change to FeedBurner effective by default, Tom pointed out that they do offer an option to turn it off.

The Quick Fix

So, for those of you using both FeedBurner and CKS:EBE, here's the scoop. On the left menu in your FeedBurner Feed Stats Dashboard, in the Services section, is an option called "Configure Stats":

When you select Configure Stats, you have a section called "Reach", which has several options. You need to uncheck the box for "Track Clicks as a traffic source in Google Analytics":

That's all there is to it! Save the settings, and everything should be back to "normal".

Of course, it would have been nice if Google had posted a more conspicuous notice that they were making this change, and where it could be configured. It would have been even nicer if they had made the change "opt in" instead of "opt out".

Nevertheless, what's done is done. You should now be able to click through from my RSS feed directly to the articles you are interested in.

I apologize for any inconvenience.

At Least They Didn't Burn the Turkey

Just a quick note before I run off for the Thanksgiving holiday (USA). If you have been a subscriber to my RSS feed, you may have noticed a problem clicking through to my blog posts lately. This is because of a change that FeedBurner made a few weeks ago. They added extra parameter information to the connection string of links back to the blog.

This is theoretically a good thing, as it allows site logging to better determine where visitors are coming from. However, this blog uses the Community Kit for SharePoint: Enhanced Blogging Edition (CKS:EBE). The way CKS:EBE handles URLs doesn't allow it to correctly interpret these additional parameters. This resulted in broken page displays. You can still eventually navigate back to the right page, but it isn't as convenient as it should be.

I have just tested a patched version of CKS:EBE to resolve that problem. While the patch for the FeedBurner problem seems to work correctly, there are significant issues with other changes to the patched build of CKS:EBE. I noticed that my tag cloud was no longer resizing the keyword links to their proportions, for example, and there were major authentication problems. These glitches are bad enough that I decided to retract the update.

I apologize for the inconvenience. Rest assured, I'll continue working on getting links from FeedBurner working (without breaking everything else).

In the mean time, Have a Happy Turkey Day!

All of Your Eggs in One Basket?

Every time a new version of a virtualization tool comes out, people get all excited about the possibility of reducing the costs of running their data centers. Most of them are thinking in terms of hardware consolidation, but virtualized environments also allow for new ways of handling resilience and recovery as well.

This is all well and good, but then you run into what I call "the new hammer syndrome". The idea is, after you buy a new hammer, everything starts to look like a nail. You keep thinking of ways to apply this new tool. Some of them are great. Others, not so much.

Virtualization and SharePoint

SharePoint, in particular, frequently seems like a ripe target for virtualization. It has a multitude of roles, which can be (and often are) distributed among many servers. Data center managers see all of this hardware and envision collapsing it onto a single box. And, SharePoint is officially supported in virtualized environments. It seems like a match made in heaven, doesn't it?

But, not so fast! Take a step back and think about what I just said. You are taking all of these SharePoint roles, and spreading them out among multiple servers. Now you want to take all of these servers and virtualize them back onto a single piece of hardware? Where is the sense in that?

Why did you build out that many servers in the first place?

Pausing while people try to gather back the pieces of their exploded heads from that bit of circular logic...

Profound, isn't it? Almost like a Zen koan.

On Over-Engineering a Solution

SharePoint will very happily run all of its roles on a single server (physical or virtual), if you want it to. So, why would you want to split the roles at all? There are really only two reasons (good ones, anyway) - performance, and resilience. (No, I don't consider being able to point at a monitoring station covering a half-dozen or more servers and saying "Look at all of the systems I manage in my SharePoint farm" a good reason.)

From a performance standpoint, some SharePoint roles are real resource hogs. The two big ones would be SQL Server and Search Indexing/Crawling. SQL Server, though not technically part of SharePoint, is hit pretty much constantly by nearly every SharePoint component, and so is almost always set aside on separate hardware. The search crawl process, though intense, is "peaky." In other words, it goes through cycles of short bursts of intense activity, followed by periods of near idleness. In comparison, the web front-end functionality is a cake-walk. A single WFE server can handle potentially many thousands of users without breaking a sweat.

In fact, a big, modern server can probably handle hundreds, if not thousands, of users even with everything except SQL Server running on it. (Naturally, this depends upon just what those users are doing.) So performance is rarely the real reason for splitting off most of SharePoint's functions, except in very large environments.

That leaves resilience. Resilience is the ability of a system to keep on going, even if a piece of it fails. By splitting the SharePoint roles onto several servers, and having multiple instances of the roles that face your users, you can create a system which can tolerate a failure of any one component with minimal short-term impact. It is possible to take this too far, however. It isn't a case of "if two are good, three must be better, and five are better still."

What good is having three or more web front end servers, when they all sit practically idle at singe-digit percent utilization - even during peak periods? Not very good at all. At a minimum, it is not a very efficient use of resources. This is the kind of thing that makes virtualization look really attractive.

Balance

So, is virtualization really the answer? Maybe. Or maybe not. Let's get back to that biggest of little questions - "Why?"

Why did you build out your farm onto multiple servers?

Did you build out this big farm because your usage is so heavy, you were stressing out anything less? Then you are almost certainly not a good candidate for virtualization. In this case, you've got your hardware optimized to its load. Virtualization is just going to add another layer, and if you're already fully loading your systems, you won't get any benefit from host sharing with other virtual servers. The only reason you might justify going virtual is to be able to quickly replicate a failed system from an image, or shift a running image onto another host. But can you handle the extra overhead?

Did you build out your farm for resilience? The minimum "fully" resilient SharePoint farm is a configuration I call "2.1+". That's two servers running as WFE and Query servers (along with Excel Services in Enterprise Edition), one application server running the Index role (of which there can be only one per SSP) and optionally running other duplicated roles as well, "plus" a properly specified SQL cluster. This configuration can handle thousands of users, even with modest (by today's standards) hardware. In fact, from a performance standpoint it is probably still overkill for most organizations. Here, you might find some room to virtualize. But be careful - you split these roles out in the first place to avoid having a single point of failure. Virtualizing them and simply placing all of the VM's on a single host eliminates that benefit.

And What About SQL Server?

I decided to write about virtualization because I've been approached several times in the last week or so with folks asking about virtualizing the SQL Server side of SharePoint. Up until now, even when virtualization has been appropriate for some SharePoint components, I've always advised against making SQL virtual. But VMWare has just introduced a new version for which their marketing message is claiming that SQL virtualization is now a good thing.

Frankly, I'm not convinced. My main concern is that SharePoint is a very heavy user of SQL Server. Again it boils down to your utilization. Are your SQL Servers already CPU or disk I/O bound? If so, virtualization isn't going to help matters. If not, then you might be OK. Even if you decide to virtualize computation, I would still avoid virtualizing the data storage without first doing extensive load testing.

Going Forward

Ultimately, all computer configuration involves trade-offs. Cost, performance, and resilience are three corners of one of those "pick any two" engineering triangle conundrums. Virtualization doesn't eliminate the trade-off, but it can shift the balance toward the lower-cost corner. Whether or not it is appropriate in your case will depend upon your server load and tolerance for risk. In the case of SharePoint, you can achieve many of the same results as virtualization by simply re-consolidating the roles that were originally split off.

Mark Twain once said: "Keep all of your eggs in one basket - then watch that basket!" If you do decide to virtualize, make sure you take appropriate precautions. The Microsoft Consulting Services UK SharePoint Team has written an excellent series of articles on SharePoint virtualization. I suggest checking that out for more technical details.

Enhancing a CodePlex Project

In this article, I'm going to get a bit more technical than is my normal wont. In fact, I'm going to go into Visual Studio, and show you some C# code as well.

(Waiting a few minutes for folks who know me to pick their jaws up off the floor, or even find some smelling salts... Yes, Virginia, I really can program when I have to. :) I just very rarely have to.)

That doesn't mean I'm going to be building this project from scratch. Far from it. So, if you're looking for info on how to build a .wsp solution package, this isn't the article for you. I am going to talk a bit about user choice, web part properties, a programming technique called "recursion", and the SharePoint API (in particular, accessing the User Profile store).

What I'm going to do is show you how to fulfill a very common business requirement by starting with a fairly simple CodePlex project, and tweaking it until it is something that does just what you need.

A Common Request

Virtually every client I've implemented SharePoint for has asked for an organizational chart. The problem is, there aren't very many org chart web parts out there for SharePoint, and those that are, tend to have issues (usually performance) - especially if you have a fairly large organization.

SharePoint Server 2010 finally makes a really cool organization chart part of the package. It is Silverlight based, and works pretty fast. Unfortunately, that's 2010, and it is going to be a while before many companies are ready to deploy it. We need something that will work for "today".

Finding the CodePlex Project

So what do you do? Well, a quick online search reveals that there is indeed an org chart part on CodePlex, called (quite originally) "OrgChartPart". This was written by Rodney Viana. Looking at the specs, it sounds promising:

• It reads profile information from SharePoint
• It uses a free JavaScript Org Chart rendering engine
• And of course, it is free, too!

What's not to like?

There is a bit, as it turns out, but let's start with the good. Installing the OrgChartPart is a breeze. Download the .wsp file from CodePlex, install it with the "stsadm -o addsolution" command, and deploy it to your web application(s). You will then need to add the part to your web part gallery (there is a feature that does this, or you can do it manually), then add it to a page. (I suggest you add it to a web part page that has nothing else on it, for reasons you will see in a moment.) This takes 5 minutes, tops.

The web part reads your MOSS profile store, and emits the chart. Just like that, you're done!

On the surface, that doesn't sound too bad. But dig a little deeper, and you find a big red flag - the author warns that you shouldn't use the part if you have more than 500 profiles in your organization. Of course, if you have more than 500, you've got a problem.

But even if you have fewer than 500 profiles, you might not like the results. Consider the following clip of the chart initially rendered by this part:

This is a pretty complicated organization - it doesn't fit on the screen, and it is completely expanded by default. But, there's another issue. The chart looks a little "top heavy". That's because the Active Directory includes a number of accounts that aren't actually users, and therefore don't have associations in the hierarchy. Looking closer, you can see that Patricia Doyle, the CEO, has conference rooms as peers! There's also some clipping of titles and departments.

Note: This is technically a problem with the Active Directory and/or its import. In theory, you could set up the AD/Profile import to exclude non-person entities. But, that's another story... :)

Looking Under the Hood

Still, even with those issues, you can see the potential in this part.

Note: Most of the remainder of this article discusses how the OrgChartPart was originally implemented, and how I updated it. If you just want the end results, you can download the original source from CodePlex, plus my replacement OrgChartPart.cs file and rebuild the solution. (I have also given the code updates to Rodney, who may update the actual CodePlex project.)

Fortunately, as with most CodePlex projects, we have access to the source code. I downloaded the source and opened the project in Visual Studio. What we see is, the part is actually quite simple. There is one .cs file (the two in the image below include my edited version, plus the original) and the ECOTree JavaScript library. ECOTree is what actually does the heavy lifting of drawing the chart. By default, OrtChartPart uses just a fraction of the power of this library (follow the link above for more details).

There is also a "departmentconfig.xml" file hiding in the Layouts folder of the 12 hive. Hmmm...

Looking at the code itself, we find there are four main functional areas beyond the basics needed to render a web part:

1. Handling that department file
2. Crawling the profile store
3. Building a profile render block
4. Building the chart

Now, We Tweak

In the rest of this article, I'm going to look at each of the sections I described above, tell you how the original code worked, and show you what I did to update them. While I won't embed the entire source in the article, you can easily download it if you want to examine everything in context.

On the Department File

For now, I'm just going to say that while the idea of mapping the department information in the profile to department home pages is admirable, the way it was implemented was causing some serious performance issues, and would be a long-term maintenance headache. Therefore, I commented out the two lines that actually make use of it (249 and 267 in my updated file), and replaced the link rendering with a simple text rendering. Here are the original and replaced lines from the updated code ( 267, 268).

//this.department = config.Find(EmptyIfNull(Profile[PropertyConstants.Department]));
this.department = EmptyIfNull(Profile[PropertyConstants.Department]);

"config.Find" is part of a config object that is defined elsewhere in the code. For purposes of this article, though, we're just going to ignore it.

Crawling the Profile Store

The profile crawl itself takes place in a function called PopulateOrg(). Examining this routine, a few things stand out.

First, there are two different ways for nodes to be added to the chart. One uses information read from a user's profile, the other feeds static information into a node. (This is used primarily to display an error message, if needed.)

Second, the population routine reads the entire profile store in one fell swoop. Combined with the inefficiency of the department link mapping mentioned above, this is a big part of the performance limiting the part to organizations with 500 or fewer profiles.

Finally, hidden in this routine (but commented out) is a nice little set of sample data.

At its core, an organization is a hierarchy. A hierarchy is made of parents and children. Technically, as long as you know the parent of each node, you can manually construct the organization. This is the approach originally taken by this web part.

foreach (UserProfile pf in pmManager)
{
     Details dt = new Details();
     dt.AddProfile(pf, this.Page.Request);

     if (dt.TrimFullName != String.Empty)
         employeeList.Add(dt);

}

However, a UserProfile object itself has methods for listing its children and ancestors directly. This means that we can easily start at any point in the tree, and generate the organization descendent from it, as well as the chain of command above it.

The ancestors are represented in a single collection, called through the GetManagers() method. The first level of children are just as easily retrieved with the GetDirectReports() method. However, if we want more than one level of child, we would then need to call the GetDirectReports() method for each child returned in the previous call.

If you try doing this with loops, things can get complicated pretty quickly. However, there is a simple way to to handle an arbitrary number of child nodes, to an arbitrary depth. This is a technique called "recursion". Essentially, you create a function that calls itself. In order to keep it from running forever, one of the parameters that should be passed to a recursive function should be a "depth" limiter. Within the function, the child calls decrement this limiter, and stop calling when it reaches a threshold level. In my update, I create the recursive function addChildren(). This takes the starting profile, and the depth limiter as parameters.

protected void addChildren(UserProfile Parent, int levels)
{
    try
    {
        foreach (UserProfile pf in Parent.GetDirectReports())
        { 
           Details dt = new Details();
            dt.AddProfile(pf, this.Page.Request, false);

            if (dt.TrimFullName != String.Empty)
            {
                employeeList.Add(dt);
                if (levels > 0)
                    addChildren(pf, levels - 1);
            }
        }
    }
    catch (Exception ex)
    {
        AddInList("1", "", "Error", "", ex.Message); 
    }
}

Notice also that I have added a parameter to the AddProfile method of the details (dt) object. This third parameter allows me to force the profile being added to act as a root, or top-level, node. (This is used for the management chain of command.)

Giving Your Users Choices

Looking at the stuff I've described in the preceding section, there is something very different from the original implementation - instead of crawling the whole profile store, I crawl a specified subsection. That requires two pieces of input - the node I plan to start with, and how deep I want to crawl.

Fortunately, when you create SharePoint Web parts, you can easily allow the user to enter configuration information. In this case, I am actually going to give the user three options. The third will allow the user to show the sample data instead of the active directory. (Naturally, I uncommented the sample population section, and wrapped the two population methods in an if...else block.)

To add a property to a web part, you add some descriptive information to say how the property is to be displayed. You also set up a public property variable to appear in the configuration pane, and a private variable to use in your code. The private variable can include a default value. The public property includes a get and a set block. While these three properties simply take the value as given, you will see later that this isn't the only option.

[Personalizable(PersonalizationScope.Shared)]
[WebBrowsable(true)]
[System.ComponentModel.Category("Org Chart Settings")]
[WebDisplayName("StartProfile")]
[WebDescription("Enter the top user to display:")]
public string StartProfile
{
    get
    {
        return startProfile;
    }
    set { startProfile = value; }
}
private string startProfile;

[Personalizable(PersonalizationScope.Shared)]
[WebBrowsable(true)]
[System.ComponentModel.Category("Org Chart Settings")]
[WebDisplayName("ChartDepth")]
[WebDescription("Subordinate levels to display:")]
public int ChartDepth
{
    get
    {
        return chartDepth;
    }
    set { chartDepth = value; }
}
private int chartDepth = 1;

[Personalizable(PersonalizationScope.Shared)]
[WebBrowsable(true)]
[System.ComponentModel.Category("Org Chart Settings")]
[WebDisplayName("Show Sample Data")]
[WebDescription("Show sample data instead of profile info:")]
public bool ShowSample
{
    get
    {
        return showSample;
    }
    set { showSample = value; }
}
private bool showSample = true;

This is how the properties look in the Web part configuration pane. The "CacheTime" parameter was included in the original part (though it is unused in the code). I'll discuss the "Left To Right" parameter later.

You might also notice that I set the default value of showSample to true. This allows the user to place the web part on the page and see a reasonable representation of its function, without needing to first set the start point and depth.

Note: The chart initially renders with only the first layer of children open. I expanded it for this screen shot.

Rendering a Profile

Each node on the chart is constructed of HTML. The HTML is assembled in the ReturnItem() function.

I wanted to enable two functions not already present - showing the user's profile picture, and allowing the user to "re-home" the chart on another node in order to enable navigation of the entire organization in convenient sections. This meant I needed to add two properties to the details object, and then add the code to display these properties appropriately in the chart.

In order to make the re-home function work, I added an override to the StartProfile. Essentially, if there is a rootuser query string parameter, the profile specified there will be used instead of the one set in the web part property. The SetRoot link calls the current page with rootuser set to the selected profile.

In addition to the actual HTML, however, the look of the node contents are controlled by styles defined in the ECOTree.css file. In order to allow the picture to render, as well as permitting the text to fit, I needed to modify the .econode class in this file as follows.

.econode {
    text-overflow: clip;
    font-family: Verdana, Geneva, Arial, Helvetica, sans-serif;
    font-size: xx-small;
        color: White;
    padding: 2px;
}

Rendering the Chart

Just as we built up a node by assembling the HTML, the chart itself is built by assembling JavaScript. One of the things I did was to examine the functions provided by the ECOTree library. I found that it offered a lot of flexibility that wasn't currently in use. I felt the most important was the ability to set the orientation. Here's where that "Left to Right" checkbox comes into play. The library uses a parameter .RO_TOP or RO_LEFT to decide whether the chart is top to bottom, or left to right.

This is the code that reads and sets that web part parameter:

[Personalizable(PersonalizationScope.Shared)]
[WebBrowsable(true)]
[System.ComponentModel.Category("Org Chart Settings")]
[WebDisplayName("Left To Right")]
[WebDescription("Root at left, otherwise at top:")]
public bool RootLeft
{
    get
    {
        if (chartOrient == "LEFT")
        {
            return true;
        }
        else
        {
            return false;
        }
    }
    set
    {
        if (value)
        {
            chartOrient = "LEFT";
        }
        else
        {
            chartOrient = "TOP";
        }
    }
}
private string chartOrient = "TOP";

This demonstrates the kind of things you can do with the get and set blocks of a parameter. The user gets to the see the checkbox that represents a binary choice, but the internal variable is a string, meaning that we can render the part simply appending the string value without inserting an "if" block.

myTree.config.iRootOrientation = ECOTree.RO_");
sb.Append(chartOrient);
sb.Append(@";

The Results

The image below shows you how the revised chart looks with the same profile store shown in the first image.

This is a much more manageable presentation. It allows the user to drill into the organization as needed, and since we set the starting node on our CEO, those conference rooms who don't report to anybody are nowhere to be found.

I hope this article has shown you how easy it is to stand upon the shoulders of giants when looking for features to add to SharePoint. I built upon the CodePlex OrgChartPart, which itself leveraged the ECOTree charting library.

There are many more things you could do with this web part. You could add options to select the color of the nodes, for example. Or give users a check-box to render the entire organization at once, as the original part did.

The options are limited only by your imagination!

Buried in Bureaucracy and Rebuffed

Note: This is a follow-up piece to my "In a Fix with Ford" article. See that if you need some additional background.

Where does one begin a tale of woe? For that, my friends, is what we have here. Six weeks ago, I wrote my original article with high hopes that I could prevail upon Ford to do the right thing, and fix the known design problems with my early production 2006 Ford Fusion. Little did I know that I had about as much chance of success as Don Quixote in tilting at windmills.

An Apology to the Messenger

First, I want to express my appreciation to Marq Boggs, Service Manager at Don Hinds Ford, in Fishers, IN. He had the great misfortune of being bearer of the bad news that Ford had no intention of fixing my car's problems. In my initial article, it may have seemed that I was "killing the messenger" by placing a big part of the blame on his dealership. That was not my intent, and if it was taken that way, I humbly apologize.

In fact, Marq has been of great assistance: first, in confirming that my car does indeed suffer from every one of the issues described in my first article; second, in taking it upon himself to address one of the problems by flashing my car's computer with up-to-date firmware; and finally, in chasing through some of the bureaucracy at Ford in an effort to find someone willing to address my problems.

Unfortunately, he was not successful.

Pointing Fingers, Hot Potatoes, and Monkeys in the Middle

The implication here is, that Ford has refused to repair my car. Interestingly, Ford customer service's own reply to me (I'll call it the "kiss-off" letter - others might choose a more colorful term) seems to imply that the dealership was the one with the final say in the matter:

Dear Woodrow Windischman,
We have looked into this matter and our records indicate that a decision has been made by your servicing dealership.  In addition, they suggested that you take your vehicle to the selling dealership and see if they would provide additional assistance.  Please be advised that the CRC can not overturn this decision.  However, to ensure our records are complete we have documented your feedback.
We would like you to know that we understand your concern and that we appreciate the time that you have taken to write us about this issue. 

(Emphasis added)

I talked to Marq again to see what was going on. It seems that each dealer is allocated a certain amount of "customer loyalty" money by Ford. This is supposed to be used to provide exceptional services to the dealer's customers. The official bottom-line position of Ford was that they weren't going to pay to fix my car, but if the dealer wanted to, they could provide the repairs out of these loyalty funds. Here's the rub - I didn't buy my car from Don Hinds, and I'm not really local (for now, I'm just in Indy on a contract), meaning future purchase prospects are limited as well. So, there really isn't a good reason for them to use a big chunk of these (rather scarce) resources on my car. I can understand this, and there are no ill-feelings from me on this account.

What about my "selling dealership" (Rice Ford, in Warsaw, IN - where I actually bought my car)? Well, I spoke to Bud Shanks, the service manager, and there was good news and bad news. The good news is, he and Mr. Rice agree that Ford should be responsible for fixing my car. The bad news is, they agree that Ford should be responsible for fixing my car. Therefore they don't feel it is appropriate to tap their loyalty funds for my repairs either. There is another complication - as I mentioned, I'm on a contract that has me over 100 miles away from the dealership during normal shop hours. In order for me to take my car there for service, I would need to take at least a day away from work, thus negating a big part of any benefit from having the repair covered.

How to Make a Billion Dollars

A company is more than its most recent marketing campaigns. I was proud of the fact that Ford didn't accept bail-out money from the government last year. I remember their "Quality is Job 1" campaign. I remember when they used to offer lifetime warranties on repair service. Notwithstanding the (truly minor) problems at issue here, I've been very happy with the service my Fusion has given me.

It is ironic that just a few days after I got their final kiss-off, Ford announced that they had made about $1,000,000,000 (that's one billion dollars) in profit during the quarter of my discontent. This during one of the worst economic downturns in recent memory. (No, how ever much the media likes to portray it that way, it isn't even close to the "real" depression of the 30's, or even the "stagflation" times of the late 70's. Doesn't anyone remember the double-digit mortgage interest rates?)

According to my experiences, and based on conversations with the dealers, a big part of this profit has been made by cutting back on the service and support provided to its customers and dealer base. Yet this "reputation" for quality and service is one of the reasons Ford was able to make sales while others were foundering.

The Big Picture

Everyone involved here agrees that it should be Ford's responsibility to fix my car. Everyone, that is, except Ford themselves.

Of course, whether paid for by Ford or not, I need to get my car fixed. This whole adventure started when I had to pay an ESP extended warranty deductible for a repair on my shifter, who's design was changed due to the very problem I suffered from. That case was made even more frustrating by the fact that the deductible was almost the entire cost of the repair. That ESP is still in place, and some of the issues I'm having are covered under that plan. Of course, I'll have to pay another deductible. In the end, given the cost of the ESP to begin with, I'll end up having paid the entire cost of the repairs, and then some.

If you've followed my blog for the last few months, you know that my life has been filled with highs and lows lately. In the grand scheme of things, is getting a few non-life threatening car repairs paid for all that important? Not really.

But that still doesn't make the way Ford has been treating its dealers and customers "right".

SharePoint Conference 2009 Wrap-up

The show is over, but the adventure is only beginning. As stated before, SPC09 in Las Vegas was the coming out party for SharePoint 2010. While we will still have to wait a few weeks for stable bits to play with, over 7000 attendees came away with a treasure trove of knowledge and documentation.

In my previous installments, I talked about the venue, the atmosphere, and the keynotes. I've also touched on some of the new Office integration story.

Of course, the star of the show was SharePoint 2010 itself. So, I'm going to dedicate the rest of this post to a punch-list of changes/improvements. I know I've missed a few (or more than a few) new elements, or misunderstood a detail or two, but even so the list is impressive. You'll see that the SharePoint team at Microsoft have not been resting on their laurels during the three years we've been waiting. Over the next few months, I'll fill in more details on the individual features, correct what I got wrong, or update you on the inevitable feature changes as things get closer to release.

The Basics

It seems Microsoft can't release a new version of SharePoint without tweaking the names a bit. Just as "SharePoint Portal Server" and "SharePoint Team Services" became "Microsoft Office SharePoint Server (MOSS)" and "Windows SharePoint Services (WSS)" in the 2007/3.0 wave, For 2010/4.0, they're simply called "SharePoint Server" and "SharePoint Foundation" respectively.

The Public Beta of SharePoint 2010 is to be released in November 2009.

The actual product release is planned for the first half of 2010.

It is still "SharePoint". Although many weaknesses have been addressed, core functionality remains essentially similar, with lists, libraries, site model, etc... Since "form follows function", many of the visual elements will be very familiar.

Infrastructure and Administration

• Requires 64bit throughout the stack
• Windows Server 2008 as a baseline OS
• Complete redesign of Central Admin
• Shared Services: No more monolithic "Shared Services Provider". Instead things formerly grouped under an SSP are individual Shared Service Applications.
• Search architecture changes: index role can be spread across multiple servers
• "Normal" SharePoint search now scale-tested to around 100 million items.
• Business Data Catalog transformed into Business Connectivity Services, and becomes part of SharePoint Foundation (no more enterprise CAL required).
• BCS info becomes available throughout the Office 2010 suite, not just SharePoint, and offers read/write capability.
• FAST Search is available as an add-on for Enterprise CAL users at per-server pricing.
• Enterprise-wide metadata support
• Lists are more scalable, and can be "external" to the SharePoint content database. Admin can set maximum returned items to prevent bogging the system down.
• Servers can be upgraded without enabling the new UI by default. Site owners can then switch over when their members are ready for the change.
• AD Group Policies can prevent installation of SharePoint on unapproved systems.
• Even more databases.
• Better auditing and reporting in-box.
• "License" logging to see which features are used.
• Allowed to read log/report database to build custom reports.

Client Facing

• UI: No IE6 support for collaboration/team sites. Can still make IE6 friendly publishing sites.
• Firefox 3.x is a Level 1 browser.
• "Accessible" CSS-based layouts, and XSLT-based list views.
• Table-based layouts are gone
• Cleaner, modernized themes.
• Ribbons are primary control mechanism, just like Office.
• There is no longer a separate basic "Wiki" site type in SharePoint Foundation. (However, there is now a publishing-based "Enterprise Wiki" site in SharePoint Server.)
• All team sites can have wiki functionality enabled, and made the default.
• Theme colors can be imported from PowerPoint themes for compliance with corporate standards.
• The GroupBoard template is available out of the box.
• List lookups can pull multiple fields
• List lookups support referential integrity (blocking/cascading deletes)
• Field validation

Social

• Major overhaul of profiles and My Sites.
• Includes "status" functionality (i.e. FaceBook/Twitter style updates)
• Unique org-chart presentation
• "Folksonomy" to support user-created shared tags in addition to Enterprise "Taxonomy" metadata.
• Can tag non-SharePoint content

Search (Standard)

• Improved handling of metadata
• Faceting (now called "refinement") is built-in
• Social input to ranking

Search (FAST)

• All standard SP Search features
• Deep refinement (polls entire result set to get actual counts)
• Concept metadata from unstructured content
• User-role tailored result sets.
• Massive scalability
• Superset of standard SP Search API
• Managed through the same admin UI

SharePoint Designer 2010

• Complete UI redesign, based on SharePoint "artifacts" rather than file structure.
• SPD 2010 tied to SP 2010. Will not work on older versions or non-SharePoint sites, and old SPD won’t work against SP 2010 sites.
• Much better Visual Studio integration – exports Solutions that can be imported into VS for both site designs and workflows.
• SPD Workflows can be independent of specific lists.
• SPD Workflows can easily be exported into either Visio 2010 or Visual Studio 2010
• Finer administrative control over what SPD users can do.
• Page model is the same, but many changes based on new CSS layouts and Theme engine.

Development

• The "platform" aspect of SharePoint receives a lot of emphasis with this version
• Use Visual Studio 2010 for full visual web part design and other SharePoint integration points
• Can install SharePoint on a client OS (Vista or Windows 7, 64-bit) for dev sandbox.
• Much better developer documentation out of the gate
• Client Side object model to make Silverlight controls and web parts easier to develop.
• REST, ATOM, and other web service interfaces fully supported.

Conclusion

In his keynote and his write-up from a couple weeks ago, Jeff Teper pointed out that the vision and purpose of SharePoint hasn't really changed much from the original 1-page proposal over 10 years ago. Yet the implementation of that vision has grown by leaps and bounds over the succeeding versions. I hope I have shown you that SharePoint 2010 will continue in that tradition. As I said earlier, I'm sure I've missed things. These were just the elements that stuck out as I was going through sessions and reading material. But I have to admit, I'm excited.

In Loving Memory of Three Wonderful People

I know you are expecting my wrap-up of the SharePoint conference here, and that will still be coming. But, for now, I need to digress. While I was in Las Vegas, I received word that my grandmother on my mom's side (my last grandparent) had passed away. This caps a series of three losses for me and my family over the past few weeks. In addition to my grandmother, Mary Cary Rudes, I lost and attended services for my uncle, Dr. Les Meyer, and my brother's wife's father, Tom Moore.

Each, in their own way, spent much of their lives helping others. My grandmother was an English teacher and patron of the arts. Uncle Les was not only a physician, but the founder of many programs that focused on helping children with orthopedic problems. Tom Moore was a man of many talents, from culinary arts, to jazz piano, which he always gladly shared with those around him.

No matter the situation, whether expected or sudden, the loss of a loved one has an impact on our hearts. The best we can hope to do is turn it into a reflection of the impacts of their lives, and use that reflection as an inspiration to share of ourselves as they shared with us.

Tom, Les, Grandma, may your journey into eternity be filled with joy and peace.

Keynotes, Sessions, and More

I can't believe how fast the time has gone. We're half-way through the Microsoft SharePoint Conference 2009 already. If you're following me on Twitter, you'll know some of the things that caught my attention as I was sitting through the keynotes and sessions. I really haven't had much time to ponder things too deeply, but there are some items that stand out.

"SharePoint is Magical"

During Steve Ballmer's keynote, this was his answer to the question "What is SharePoint?" Although that seems more than a little hyperbolic, in some ways he's right. Not so much the product itself (though I am beginning to wonder), but the effect it is having.

Consider this: The SharePoint Conference has over 7,000 people participating. That's more than attended TechEd this year. TechEd is covers all of Microsoft's products and technologies. Yet this conference, for a single product, is bigger.

But, that is almost a mischaracterization. Although SharePoint is a single product (well, family of products), its impacts are far greater. Almost every other product Microsoft produces is influenced by SharePoint. In fact, many of the sessions at the conference revolve around some of those impacts.

SharePoint is Pervasive

One of the sessions I went to yesterday demonstrates this clearly. Access has been Microsoft's "end user" database product for well over a decade. It allows users to easily create sets of tables, forms, and reports.

With the 2010 release, Microsoft is introducing Access Web Services. Essentially, this allows you to take an Access database and convert it (not merely upload it) into SharePoint site, with virtually all of the functionality intact. Tables are converted to SharePoint lists. Macros are converted to JavaScript and SharePoint workflows. Even your forms and reports display virtually identically.

What's more, this isn't a one-way ticket - it is a round-trip. You can re-open the site in Access to tweak it, or have full "access" to the functionality that doesn't convert (either by product limit or your choice).

Integration now also exists for Word, Excel, PowerPoint, OneNote, and Visio.

"The News"

Of course, no Microsoft event would be complete without the big Attendee Party. This year, the venue was pretty close to "home" - Mandalay Bay's "Beach". The theme was the 80's, so we had breakdancing, "big hair", leg warmers and headbands, even Rubik's Cubes. About the only thing missing was roller skates.

But the highlight of the night was a private concert by Huey Lewis and the News. Just to show you how out of touch with pop/rock music I am, I didn't even know he was the guy who did the songs in Back to the Future (among other things). I only knew him from the karaoke movie, Duets, with Gwyneth Paltrow. (I'm a karaoke singer/fan.) Of course, I had heard of him, but I didn't know which music he was associated with. But I definitely recognized most of the songs he did, as did the crowd - many of whom (myself included) ended up wading into the beach pool to get closer to the stage.

The "Other" News

Of course, as you probably have heard from everywhere by now, this conference is essentially the "coming out party" for SharePoint 2010. But the actual bits aren't quite ready for us yet. They've promised to have the public beta available sometime in November.

So, that's it for me at midway. Keep up with my tweets for in-line learnings, and I'll be back with a conference wrap-up once it is all over.

Day 0.5

<cue Film Noir music>

The day started clear and cold. I would say "bright" and cold, but it was only 4:45 AM when I left for the airport, so it was dark. There was frost on my car windows as I made my way to the airport.

Traffic was light, but construction was heavy around O'Hare. The security line was long. Fortunately I had checked in online, so I didn't have to wait for my boarding pass. Still I ran through the B/C tunnel and barely made it before they closed the door.

<end music>

And so began my trip to the SharePoint Conference 2009 in Las Vegas. The flight itself was pretty much uneventful, though once I arrived in Vegas I ended up captive in a shuttle while they waited for enough people to show up to fill it.

This is my first time in Las Vegas. Although the conference is officially in Mandalay Bay, most of the hotels on "the strip" are interconnected. I, along with lots of other SharePoint folks, are staying in the Luxor. That's the one that looks like a big black glass pyramid. On the inside, that pyramid is totally hollow. The rooms are arranged in a single "layer" along the wall, so that they have steeply slanted windows looking toward the outside. Inside, on each floor there is an open hallway that looks to the hollow interior. The base is filled with restaurants, theaters, and of course, the casino.

The elevators run up the support columns in each of the four corners. Although they are sealed so that you can't see it, they actually travel "slantways" as they go from the ground floor to the peak.

Getting your Exercise

While the Luxor is nominally "right next to" the Mandalay Bay, and they are connected by an interior walkway, that doesn't mean that they are truly "close". In fact, to get from the Luxor to the conference center, you need to walk a seemingly endless maze - through both hotels' casinos as well as a shopping center. All in all, it takes at least 10 minutes, if not a bit more, to cover if you maintain a brisk pace. Let's just say I'm glad I decided to bring the shoes that are most comfortable for walking rather than sitting around.

The Registration Packet

This year's registration premium is a simple spandex laptop pouch (in your choice of sizes). But hidden inside is something very juicy - a very well prepared booklet describing SharePoint 2010's features and functions (as they exist in current beta builds). I haven't had much chance to read it yet, but let's just say it looks like Microsoft has taken to heart the complaints of SharePoint 2007 not having enough documentation available early on. They are clearly taking steps to ensure that complaint isn't repeated.

Unfortunately, SharePoint 2010 software itself wasn't in there. But, there are still 4 days of "real" conference yet to come, so who knows?

Meeting and Greeting

The great thing about events like this is getting to see, in person, folks you normally only interact with online - or at events like this. ;) In addition to bumping into people throughout the day, you can always be sure of a nice mingle or two. Sunday night, there were two big ones - SharePint, and ShareIndian. Unfortunately, I hadn't gotten much sleep over the past three days, so I wasn't really feeling up to leaving the complex, so I missed ShareIndian. I did make it to SharePint briefly, however, and had a number of nice chats before I turned into a pumpkin at around 9:30. (I suspect I was turning into a virtual vegetable long before that.)

So ends my report of the Pre-conference. Stay tuned for more throughout the week!

Preparing for the Flood... of Information

Before I get all wrapped up in the excitement around Vegas, the SharePoint conference, and the release of information about SharePoint 2010, I thought I would point out a few things.

As I mentioned in a previous post, I try to keep this blog unique. That said, with so much new information coming out, so fast, and publicized by an unprecedented number of people, there will be virtually no way to ensure I'm not repeating information. So, in this rare instance, I'm not even going to try.

I'm going to post about what I consider (and I hope you will find) interesting. Right now, I can't even predict what that will be, but you can rest assured that I'll be doing it from my own perspective. Even when I cover topics I know a zillion other people will talk about, I'm going to try to bring you something unique.

Like virtually everyone else in the SharePoint community, I'm also on Twitter. I may not be as prolific as some, but like here, I try to do things with my own spin. (But don't worry - I won't be tweeting the entire conference in Haiku form!) So if you aren't already following me, "now would be a good time." :)

See you in Vegas!

"2.1 Billion ID's Should be Enough for Anybody!"

One of the more infamous stories about Bill Gates is that he once said "640K of memory should be enough for anyone." That wasn't true - he never said it, but it did point up the frustration that came from one of the design limits of the original IBM PC. The memory between 640K and 1MB (which was the physical limit of the CPU) was allocated by IBM for video, I/O buffers, and lots of other "housekeeping", and therefore couldn't be accessed by DOS. This was fine at the time, when the typical computer came with 64K of RAM, and even expanding to 512K was a luxury; but when applications (like Lotus 123, dBase III, and even Windows itself) became complicated enough to require that memory, and more powerful CPUs became available that allowed access to even more, that big "gap" before getting to the extended memory required more effort to program around than anyone could have predicted. (Yes, that's way over-simplified, but it is enough to get the point across...)

The reason I bring up this little history lesson is to point out that when you are designing products, you have to set limits somewhere. Sometimes these limits are intrinsic, like the 1MB maximum RAM of the 8088 CPU. Others are compromises, like how much of that 1MB to allocate for system housekeeping, and where to locate it in the address space. You hope you set these high enough that most users will never see them, but they are there.

SharePoint also has a number of limits. Most of them are well documented. Some of them are "soft" limits - places where you see performance degradation. Others are "hard" limits, like the maximum size of an integer value. But some limits are buried under the covers, because they are internal to a function, and users never see the processes that are impacted. If they are set high enough, the users will never even know they exist.

Crawling Forward

Unfortunately, there is a limit that wasn't set high enough. This was buried deep inside the MOSS and MSS search databases. Most database tables have a field for a unique identifier. This is automatically incremented every time a new row is added. Typically, a SQL Server Integer (int) is used for this ID, allowing up to just over 2 billion items to be added (2,147,483,647 if you must know). That's a lot. But this value just goes up - it isn't decremented if you delete a row.

In the SharePoint Search DB, there is a table that keeps track of all of the links in your crawled content. Whenever you do a new crawl, rows are added to and deleted from this table. This table originally used the int referenced above for its ID field. Now, there can be a lot of links in a SharePoint site, but still, 2.1 billion should take an awfully long time to reach, and in most cases it does. But reach it you can. For very large and complicated sites, if you do a full crawl every day (which deletes and replaces all of the link references) you can reach it faster than you might (and the developers did) think.

So, what happens if SharePoint actually hits this limit, and runs out of IDs? It isn't pretty. Essentially the crawling process gets stuck. It asks the database for permission to write the next available row, and since there isn't an ID that can be given to it, the database just says "no". Unfortunately, SharePoint doesn't take no for an answer, and keeps asking. You will, occasionally, see an error in the event log talking about a SQL Identity failure, but unless you were aware of this possibility, it wouldn't make much sense.

Recovering

This also prevents you from effectively controlling search. Because SharePoint insists on finishing the last thing it was doing, you can't stop the crawl. Because there isn't much to go on in the logs, and it takes some SQL Server proficiency to accurately diagnose the problem, many times, this results in folks rebuilding their SSP, with all of the pain and agony that entails, just for the want of an ID.

Note: At this point, you need to consider the search index on this SSP corrupt. There is nothing that can recover the ability to crawl new content without resetting your index and doing a full crawl as described in the prevention section below.

Even if you can successfully diagnose it, there are very few supportable solutions that *don't* involve rebuilding the SSP one way or another. Remember, directly modifying the SharePoint databases yourself can result in an unsupported state. So, if you reset the seed of the maxed-out table to 1 in order to get control of the crawl back and stop it, you should restore the search database from a backup to reach a production state before you reset the crawled content (see below), which resets the database to an initialized state.

You can also restore your whole SSP from a backup, but that's almost as much fun as rebuilding it, and it assumes you have a restorable backup of your SSP.

An Ounce of Prevention

Obviously, it is much better to prevent this problem from occurring in the first place than to try recovering from it. There are a couple ways to do this. The first and best is to upgrade your SharePoint environment to Service Pack 2. Among the many enhancements in SP2, the ID fields in the search databases that were prone to maxing out are updated to "big" integers. BigInts are twice the number of bits as regular integers. That doesn't just double the capacity, though. It makes it 4 billion times as large. (For those who really need to know it makes the number of possible ID's 9,223,372,036,854,775,807!) So, if it took 6 months to reach the old limit, it would take 24 billion months to reach the new one.

If you can't upgrade to SP2, you should consider adding a periodic reset of the index into your maintenance plans - especially if you have a very large corpus, with lots of links. The option to do this is available from Quick launch in the Search Administration page.

Resetting the crawled content doesn't impact your settings, keywords and best bets, etc... But it does delete your existing index and completely resets the search crawl database - including the table ID fields. After the reset, search results will not be available until a full crawl is performed, so you should schedule this to take place during a down time and/or notify your users of the search outage. If you have multiple content sources defined, you will need to crawl all of them.

When you select reset, you will get a screen asking if you want to turn off search alerts during the reset. It will default to being selected, and you should leave it that way.

The alerts can be reactivated once your crawls have been completed.

Conclusion

As Clint Eastwood once said as Dirty Harry, "A man's got to know his limitations." Everyone, and every thing, has limits.

Limits are only a problem when you don't know about them, and don't take them into account. SharePoint, as powerful as it is, has plenty of them. In addition to the hidden limit I covered in today's article, you might want to review some of the more well known limits in the SharePoint planning material: Planning for Software Boundaries.

On Monday, October 5th, the Federal Trade Commission (FTC) published some new regulations guidelines with regard to bloggers and product endorsements. While I'm certain they will be challenged, possibly even successfully, for now they are what they are.

What does this mean for The Sanity Point? For now, not much. I'm a SharePoint guy, and pretty much everyone knows it. I like SharePoint and I always have. Besides, I very rarely "review" or "endorse" products at all. I normally just provide information to help you figure out how to make them work (or work better) if you've already purchased them. I occasionally comment on the news, or my personal life.

But this rule change does point out that until now I haven't put forth any kind of statement with regard to policy on this blog. So, with a nod and an apology to the Public Radio program called "Whad'Ya Know?", I now list my own "Four Disclaimers" for this web site.

1. These Opinions are My Own, and They are Only Opinions

The opinions expressed on this web site are my own. They do not necessarily reflect those of my employers, clients, friends, neighbors, or the dog down the street. They are not the "Official Word" of any company, unless I am quoting from their own information. Such quotes will be clearly identified.

I am not a Doctor, Lawyer, Financial, or Menu advisor, nor do I play one on TV. Any statements made on this site are those of an informed layman. Even "best practices" are not always the best in every situation. Statements that are within my field of expertise, which describe what I would generally consider good practices, may not apply in your particular circumstance. Please make complete and verifiable backups and confirm your situation with your own local resources before taking "irreversible" actions.

I employ the Akismet spam blocker, and moderate the comments on this site. While I won't usually block a comment from someone who is critical or disagrees with me, I will not approve obvious spam, repeated posts, comments which (solely in my opinion) are in bad taste, or marketing posts that simply refer someone to their own product site. Comments that are approved may be edited if required to reflect the above-listed standards.

2. I Don't Do Paid Reviews, but I Do Accept Legal Free Stuff

While I very happily share my experience and expertise on the products I'm familiar with, nobody pays me to write the stuff on this site. In fact, nobody pays me anything at all for anything on this site, except copies of my book. If I like something, I'll say so. If I think something could be improved, I'll say that too.

That said, in IT, companies are always providing trial and demo copies of software, sometimes for download, often by the stack at conventions and other events. Sometimes these are full copies, other times they have timeouts, or functionality limited in other ways.

If I find such a product useful, I might write about it. Whether I paid for it or not, I always get it through some legal channel. I do not break my own NDA's, and I don't attempt to circumvent those of others by getting software through a back-channel where the NDA isn't presented to me.

I don't run random ads on my site. I link to Amazon so you can purchase copies of my book. I wrote the book - of course I benefit from you buying it! I also have an Amazon a-Store. Products purchased there can result in a commission for me. While it isn't linked from this site right now, if I do link to it, understand that all selected products in it are those that I would recommend to anyone, no matter where they might purchase them.

The MVP Award I just received does come with some goodies that could be considered "substantial"; however, it is not based on product endorsements. Rather, it is a reward based on recent past exceptional community participation, not a payment for any particular service or statement.

3. The Sky is Blue, Birds Fly, and Fish Swim. It's What They Do. Get Over It.

4. I Value Your Privacy

This site employs conventional web reporting technologies, such as logs, and scripts from Google Analytics and Feedburner. Google's privacy policies apply to their services. By the very nature of the Internet, information available to all web sites (not just this one) includes visitor source IP addresses, objects viewed, browser capabilities, session cookies, and several other types of information, all of which may be included in these logs and/or passed on to the analytics service. Other than this, and the information you enter into comments, I don't collect potentially personally identifiable information. I don't sell even the little information that is collected to anyone.

Last week, I got some great news: I have been awarded the Microsoft Most Valuable Professional (MVP) for 2009!

To quote the Microsoft MVP site:

"Since the early 1990s, Microsoft has recognized the inspiring activities of MVPs around the world with the MVP Award. MVPs freely share their deep knowledge, real-world experience, and impartial, objective feedback to help people enhance the way they use technology. Of more than 100 million users who participate in technology communities, around 4,000 are recognized as Microsoft MVPs."

Those 4000 awardees are selected from across a variety of disciplines, specializing in a wide array of Microsoft products. Only a small fraction of them are SharePoint MVP's. This is not the first time I have been given this award, but it has been a few years. (A person's community contributions are analyzed each year - there is no "automatic" extension.)

I am deeply honored to have once more been selected for this group. Thank you, Microsoft! I hope to continue providing the SharePoint community with as much help as I can over the next year.

Complete Agreement

As you may have noticed, I try to keep this blog "original." In other words, I don't cover the same material as another blogger has recently done, unless I have something major to add, or there is some kind of debate going on.

Today (September 29, 2009) SharePoint MVP Bil Simser has managed to "scoop" me, by writing an article on almost exactly the same subject I was planning to comment on in an upcoming post. In it, he talks about people who claim SharePoint is bad, and questions how whatever the "SharePoint Killer" of the day might actually compare beyond one particular forte.

While I would have been a little softer in my language, I can't fault the sentiments he has expressed. So, rather than seem like I'm repeating it, I'll simply give you the link to his post:

SharePoint FUD... Spreading Far, Wide and Fast

Note - I have actually written on this topic before, though specifically on the subject of Wikis. For my take on that subject, check out this article, from April of this year:

Wiki-in-the-Box - Is SharePoint Wiki Really that Bad?

Be sure to check out Bil, and all the other great SharePoint bloggers on my Blog Roll...

Helping SharePoint Help You

A SharePoint system manages a huge amount of data. Amazingly, in a SharePoint content database, all of the data, for every list and library item, in every site and subsite, is stored in a single table. Looking at hundreds of sites, each with dozens of lists and libraries, each potentially containing hundreds or thousands of items, and you end up with one massive table!

Ask any DBA how to achieve maximum performance on a huge table, while other options may also come up, at a minimum you'll always hear the word "indexing". And make no mistake - SharePoint (whether Windows SharePoint Services 3.0, or Office SharePoint Server 2007) does do a lot of indexing. But that is only dealing with the user data table as a whole.

Once SharePoint has figured out which site and list the data belongs to, normally it is pretty much done with indexing. When you perform a query - whether in code, or for a web part view - each item in the list is examined individually for a match. As the amount of information in your sites grows, this can take quite a bit of time and cause significant slowdowns (This is independent of the "2000 item limit" - see sidebar).

Fortunately, you don't have to put up with this default behavior. SharePoint gives you the additional ability to index the information within individual lists or libraries.

Look at the settings page for just about any list or library, and you will find a link for "Indexed columns":

When you click the link, you will be given the opportunity to select which columns in your list you wish to index. This is where an understanding of your information, and how you use it comes into play. While you could just click everything, that isn't usually a good idea. For each column you index, SharePoint needs to store extra information about every item in your list.

You should only select columns for indexing that you will be using to query/filter, sort, and group your list. For this list, I'll usually need to do this with the item's title (or name), who created or modified an item, and when it starts. So those are the columns I'll select to index:

Note: When setting up indexed columns, you will almost always want to include the title or name field.

Once you click OK, SharePoint will build the appropriate extra indexes. While there won't be any change to the way your list looks, you should see the performance results almost immediately. (Of course, the more items in your list, the greater the impact will be...)

One place you usually see immediate results is when you click on the context menu of a column title to change the sorting or filtering. The list of unique values builds much faster on an indexed column.

That's all there is to it! Setting up indexed columns in your SharePoint list really is that simple. Give it a shot, and you might be surprised at how much faster your SharePoint applications can be.

"Who Did You Say You Were?"

Today I'm going to talk about user accounts. In particular, Windows Active Directory accounts. You might be thinking, "This doesn't sound like a SharePoint topic!" But rest assured, it is.

I've talked about Windows accounts and SharePoint before. For example, I've told you about the many accounts you need to consider when setting up a SharePoint farm, and how to discover the Setup User account after the fact. I've also mentioned how you can let SharePoint know when User ID's change.

Normally, each person has one Windows account (user ID and password). They use this account to log into their PC in the morning, thus proving to the network who they are. This is called "authentication". Many systems that recognize Windows authentication (including SharePoint) will simply accept these credentials from the user's PC, without any further user intervention.  The systems use these credentials to determine what data and functions the user has been "authorized" to access by the administrator of the system.

Note: Behind the scenes, it is quite a bit more complicated than that. While a complete discussion of handshaking and protocols is beyond the scope of this article, understand that the negotiations taking place do result in one of the issues I will describe below.

On Being a Highlander - "There Can Should be Only One"

By and large, this system works pretty well. However, it is based on a pretty big assumption - that each user has only one account for their "day to day" system usage, and each account has only one user. Unfortunately, from time to time this assumption doesn't hold true. This can cause some subtle, and not-so-subtle (but hard to trace) problems in SharePoint if you aren't careful.

I'm going to discuss three main scenarios:

1. One user has multiple accounts
2. One account is shared across multiple users
3. Accounts in different domains that have the same UserID portion, but different passwords.

For each group, I'll talk about why it might occur, how it presents a challenge, a few variations on the theme, and what you can do to minimize the difficulties presented by it.

One User, Multiple Accounts

Why it might happen:

There are many specific reasons a user might have multiple accounts, but they generally fall into three categories - Administration, Test, and Transition.

Transitions can be wide-spread - such as when companies realign or change naming standards, or individual - such as when marital status changes. Regardless of the reason for a transition though, generally there will be an "old" account, and a "new" account.

For purposes of this section, I'm considering administrative and test accounts to be equivalent. They don't need to be "administrators" per se, and the account may exist mainly for applications other than SharePoint. Essentially these are any accounts that a user signs into for a particular task that has different privileges from their normal User ID (e.g. DBA), and their use is often dictated by policy or best-practice. Regardless of the reason, the challenges and resolutions are basically the same.

Why this is a challenge:

Aside from the obvious - making sure each account actually has access to the correct resources - SharePoint keys a lot of stuff based on the current user. From Created and Modified by tags, to personalized pages, and audience targeting, SharePoint knows and shows who you are. But the real hazard of multiple accounts is their effect on profiles and my-sites.

User profiles are crated from Active Directory imports, as well as user-entered information. Certain features of My Sites, such as organizational relationships are built using the imported information. Typically a user's "primary" account will populate such fields a their title, office location, manager, and other useful business information. Administration and Test accounts, on the other hand, generally just use the name to describe the purpose of the account, and little else.

These impact not only the current user, but others as well. For example, suppose you fully populate AD info for each of the accounts. When someone clicks on the manager's profile, suddenly they will show all of the secondary accounts, as well as the "real" user, in their reporting relationships. Or, what if the user has reports of their own? Since each can only have one manager, you need to be very careful to assign it to the correct account, otherwise, someone may end up not appearing in org charts, or they may appear in the wrong place.

When someone starts using these secondary accounts for day-to-day activities in SharePoint, this is the account noted as the creator or modifier of data items, and tracked in logs. If you use Communicator, SharePoint's presence indicators can also be affected. They will show the presence and contact info for the account that actually made the change, rather than the potential real presence for the user.

Finally, aside from the profile, each of these accounts will register as separate for the creation of My Sites. Since Office can hook into a user's My Site as a default storage location, this can also cause confusion, as personal documents appear and disappear based upon which ID user is using to perform their activities.

Minimizing the pain:

Wherever possible, strive to make transitions instantaneous - at least as far as SharePoint is concerned. Don't let users access SharePoint with more than one of these accounts at a time. When the time comes, make use of the stsadm "migrateuser" operation as soon as possible, so that users don't get confused or accidentally start generating content under the new account before their old information is reassigned.

For admin/test situations, make sure there is a clear distinction in the users' minds about the purposes of each account. In addition, make sure the metadata in Active Directory correctly reflects which account will be used for day-to-day operations.

One Account, Multiple Users

Why it might happen:

The reasons for multiple users sharing an account typically revolve around cost savings in one form or another - either licensing or administrative.

A typical example might be in a "shop" situation, where one PC is on the floor, and once it is logged in, everyone just accesses the information they need.

Why this is a challenge:

Some of the challenges here are similar to those you might face in an Internet-facing, anonymous access, scenario. Essentially, when someone does something, you don't know who it was.

But there is an added complication. When you are authenticated, this triggers some things in SharePoint. For example, since you have an account, SharePoint will treat you as a named user for "Created by" information. But you can't use effectively use the "only their own" global list permission, or filter things by [Me], as everyone who shares the account will have that permission or see that information. Or on surveys that prevent multiple entries, only one person will be allowed to fill it in from that account. Discussion comments all appear to come from the same person.

Minimizing the pain:

By sharing a single account, you essentially remove the effectiveness "social" elements of SharePoint. Consider reducing confusion by turning off access to MySites and Personalization for the shared account. If you want to use interactive discussions, add a field for users to manually enter their names when posting, and make it a required field.

Frequently such shared resources are "read only". Consider simply allowing anonymous read-only access in those instances.

Another option is to enable a forms-based authentication zone for that group. This allows you to keep your AD clear of staff who otherwise don't use PC's, but still maintain individual control over SharePoint access, and monitor who is posting to writable areas.

Name's the Same, Different Domain

Why it might happen:

Unlike accounts within the same domain, which by definition must have unique IDs, it is possible for accounts in different domains to have the same "userID" part. For example, EMEA\SallieJo, AM\SallieJo, and APAC\SallieJo.

These may represent different users, or one user who travels. Sometimes organizations merge, and each already has their own Active Directory domain. Or for various reasons, your organization requires multiple domains on an ongoing basis. In some cases, this can be similar to "One User, Multiple Accounts". In fact, sometimes the situation is the same - accounts are held by the same user, and/or they are transitional. If this is the case, then the cautions mentioned in that section apply as well; but this situation presents its own unique challenges.

Why this is a challenge:

Here is where that "Behind the scenes" techie stuff I mentioned at the beginning comes into play. Essentially, when a user's web browser (Internet Explorer) and IIS (Internet Information Services) start negotiating authentication, only the UserID portion of their credentials, along with encrypted password information is sent from the client to the server. (SharePoint relies on IIS and the ASP.NET engine for authentication.)

Here's the problem - IIS usually assumes the user will be in the same domain as it is, and will try to match the ID with an account in that domain. Only if it doesn't find a match will it make a deeper query of Active Directory. If it finds an ID match, it will try to validate the encrypted password against the current domain. If the passwords match, no problem - or maybe big problem. SharePoint thinks IIS has authenticated the local user. If both accounts are really for the same person, you're OK. If it is a different user from the other domain, who just happened to have the same ID and password, they could actually be seeing the local user's information!

If the passwords don't match, unlike the situation when the UserID is different, IIS won't continue its search. It will just return a fatal error - typically a "500". (If the user is persistent, this also has the potential of locking out accounts in IIS' local domain.)

Note: This isn't an issue unique to SharePoint - you can encounter this problem with almost any system that requires NTLM pass-through authentication.

Minimizing the pain:

The best defense here is to make sure your user ID's are unique across all trusted domains accessing resources. Before initiating trusts, run reports and reconcile them, and set forest policies to prevent duplicates.

When duplicates exist for one or two users, a workaround for users in the remote domain is to assign your site to a specific zone (e.g. trusted sites) and configure Internet Explorer to always prompt for authentication in that zone.

If this is a problem for a large number of users, configuration changes at the server may be in order.

When the majority of your users are in a domain other than the one hosting SharePoint, you can configure IIS to use digest authentication and a different default realm. You can also extend SharePoint into multiple zones, and configure a different realm for each site in IIS. Then ensure that each domain's DNS points to the correct SharePoint zone.

Of course, if you change configurations - whether they be IE or AD, IIS or SharePoint - make sure you document them!

We interrupt this blog to bring you a small consumer rant. If you have no interest in cars or complaints, please disregard this post.

On Being a Ford Guy

Let me start this out by saying that I'm what they, in a past age, would have called "A Ford Man". I've owned a lot of cars over the course of my life, and most of them have been by Ford Motor Company (memorable exceptions include my first car - a 67 Dodge Dart - and the dilapidated VW Beetle I had in college, but didn't everyone have one of those at some point?). This includes every one of the four cars I have purchased new.

This doesn't mean that I wouldn't consider other brands - I do a pretty thorough search and comparison before major purchases. It is just that when new car time rolls around, more often than not I find that one of their products best meets my needs. And if it is otherwise a tie, the Ford (or Mercury, or Lincoln) product will probably win by default.

Falling for the Fusion

A few years ago, Ford decided to split one of their best selling car lines of all time - the Taurus - into two: A slightly larger car called the 500 (which has since reclaimed the Taurus name, and just received a major makeover); and a somewhat smaller vehicle called the Fusion (for those reading this in Europe, the US Fusion is nothing like the car of the same name sold over there - the US car is more of a size and kind with the current Mondeo).

By sheer coincidence, the Fusion came out right at the time I was ready to replace my old car (a Focus). It was virtually everything I was looking for in a car at the time. Now, that doesn't mean I was "first in line" to buy one. I still did all of the appropriate comparisons, but buy one I did, and it is one of the earliest made (November 2005, according to the sticker - more on that later). I am now the owner of a 2006 Ford Fusion SE, in "Red Fire":

In general, I have been very happy with my Fusion. It is comfortable, roomy, performs well with the 4 cylinder engine, and gets great mileage (well over 30 MPG on the highway). And, until recently, I haven't had any mechanical problems, either.

In the Driver's Seat

I don't buy a car to just look at. I buy it to drive. A lot. When you hear about warranties for "X years or Y thousand miles", you can bet your bottom dollar that I'm going to hit the miles number first. I spend a lot of time behind the wheel of my cars, and I get to know them pretty darned well. I learn every every quirky noise, every bump in the seat, every crease of the dash, on a first name basis.

I've never gotten rid of a car with less than 6 figures (100,000 miles) on the odometer. I understand the concept of "normal wear-and-tear" as well as (if not better than) anybody. I accept them as a fact of life, and if my issues were in that category, there wouldn't be a need for me to write this article. In addition, I also always buy the "best" extended warranty plan available from the manufacturer (Ford calls this their "ESP") to make sure I get to at least 100,000 miles without a major breakdown expense. Until now, I have not regretted it.

"You Will Know a Pioneer..."

I also understand the realities of product design and manufacturing. Nothing is perfect out of the gate. Issues are discovered, and changes are made. Most of the problems are caught early. Many are fixed at the factory before the cars are delivered, and 99% of the time, this fine-tuning goes unnoticed by the consumer.

But occasionally a problem isn't caught, or a design flaw that impacts durability can't be discovered, until well after the cars are sold and people are using them in the "real world". Again, most of these issues are benign, and may never be noticed (e.g. a piece of misaligned trim on the bottom of the dashboard). Now and then, though, one rises to the level of consumer awareness.

In the case of problems that could affect safety, depending on how wide-spread it is, a general or specific recall is issued. Everyone impacted is alerted, and their cars are fixed. Period. Again, if my issues were in this category, I wouldn't be writing. Or would I?

And what about problems that aren't generally safety related? These aren't ignored, but they aren't exactly widely publicized either. They are handled through a process called the "Technical Service Bulletin" or TSB. Sometimes these are referred to as "Hidden Warranties". Essentially, they only get fixed if the customer notices and complains about the specific issue covered. Of course, if you are within your normal "bumper to bumper" warranty period, this won't even be an issue. It will be fixed no questions asked.

But that would be the case even if there wasn't a "known problem" with the design, and it was simply a one-off defect in material or workmanship. A TSB might be issued with with a symptom description like "premature wear". And when the fix involves a redesigned component, you can be pretty sure that the original design itself was faulty.

A "Key" Problem

Here is where my relationship with Ford starts to get a bit shaky. As I mentioned, my car was one of the first Fusions produced. Sometimes these design flaws take a while to manifest, and detecting the problems tends to be time, not mileage sensitive. But I drive a lot, so I hit the standard warranty mileage cap ages ago.

A few months back, I ran into a problem where my key wouldn't come out of the ignition. These days, leaving one's key in the car is not the wisest of actions, so of course I went to a dealer to get it fixed. Upon diagnosing the problem, the dealer noted that the issue and solution was described in a TSB, but because I was out of standard warranty by mileage, I would have to pay the ESP deductible of $100. (All the more frustrating because the repair itself would only have been within a few dollars of that).

While I had been aware of TSB's, this incident brought them to the forefront of my attention. First, I became upset that I had to pay at all to have what was clearly a design flaw remedied. Second, I was more frustrated because when I go in for ANY service, I regularly asked if there are any outstanding recalls or service bulletins that apply to my car.

Chasing the Wild Goose

So, I did what any self-respecting, Internet-savvy geek does these days - I Tweeted about it. I caught the attention of Ford's Social Media Guru, Scott Monty (@ScottMonty), who brought me into contact with Shawn @FordCustService. After taking my issue offline with Shawn, he eventually escalated me to Joe Wiegland, Program Manger for the ESP program. While he wouldn't do anything about getting my already-paid deductible refunded, he did ask me to contact him when I was ready to have some other chronic issues, that I had chalked up to either "facts of life" or "normal" wear and tear but my new research identified to be TSB items, addressed, and he would get things straightened out.

Here I must note - I appreciate Mr. Wiegland's involvement, but I believe this shouldn't even have been made an ESP issue. These are design flaws, not strictly material and workmanship defects, and should be repaired regardless of warranty status - normal or extended.

For various personal reasons, I was unable to return to these problems until a couple weeks ago. I called Mr. Wiegland's office, but was directed to his voicemail. I left a message reminding him of my situation, and requesting a callback.

That callback never came.

Nevertheless, armed with my list of applicable TSBs (see below), I paid the dealer (Don Hinds Ford, in Fishers, Indiana) another visit. For every issue, I was rebuffed. Frequently, it was "cannot reproduce". But even when acknowledging a problem, they replied with either "not covered", or that it would only be "partially" covered, even if they could reproduce it.

So, I took back to Twitter. Just a few posts saying that I wasn't pleased with Ford. This time, I got the attention of the newly created @Ford corporate account. They said, literally: If we had a clue as to why, we might be able to help. So, I followed them, and when they (presumably auto) followed me back, I provided contact information via a Direct Message in order to discuss it offline.

They never responded.

A Blogger Scorned

I had no desire to write this article. I tried to resolve my problems privately, but didn't get any satisfaction, so I'm turning to the bit-stream. I've laid out my story in language plain enough for anyone to understand. I'll bring it to the attention of the various Ford accounts on Twitter. I will update it with any resolution - or lack thereof -that may be forthcoming.

Note: I still like Ford products in general, and my Fusion in particular. I just want it fixed.

Appendix - The Issues

Here's a list of my issues, their TSB numbers, the dealer's response, and a few comments.

Key stuck in ignition / Shifter binding condition

My listing shows it as TSB 07-2-1, but the dealer worksheet shows 07-14-7

This is the one that started it all. The title is self-explanatory. The repair consisted of replacing the shifter handle with a redesigned unit. I got charged a deductible that I don't think I should have.

2.3L Engine belt squeal - verify proper serpentine belt and tensioner

This is TSB 06-22-13

While I was at the dealer the first time, I also mentioned a pretty noisy squeak/squeal that I get when the car is cold or the air is particularly damp. Unfortunately, the car is warmed up by the time I get to the shop, so the dealer couldn't reproduce the problem.

After I came in with my list of TSB's, however, this was the one where they said even if they could reproduce it, because the repair incorporated a part the ESP doesn't cover (a belt, which is usually considered a consumable), there would be a significant charge.

According to one source "Ford has issued a newly designed shield, belt, and tensioner to correct the squealing problem. Alignment instructions for the new tensioner are included in the TSB"

Redesign a part to correct a problem, the original design was bad. The belt is part of the redesign, and therefore should be covered.

Since it only happens when cold, here's a video I made that illustrates the noise. I kept the volume of the video down so that it isn't too bad in an office. Rest assured, that in person it is VERY loud, and I can be heard coming down the street from a block away.

Video Demonstrating Squeak and Window Issue

Note: this video also includes a segment demonstrating a crunching noise made by the driver's side window. As far as I know, there isn't a TSB on that, but it is also an intermittent issue that I wanted to get on record so maybe I can get it fixed before my real ESP extended warranty expires. My gut tells me this is a power-window failure waiting to happen. Unlike the squeaking, it is most obvious when it is hot out.

Seat Bolster Wear

This is TSB 07-13-3

Here's one that really bugs me. I brought this up well over a year ago to my local dealer's service department. At the time, it hadn't yet worn through the fabric. They said that it was normal wear that they couldn't do anything about. Now I have discovered that it is anything but.

The TSB description lists the cause as a wire inside the bolster cushion that causes "premature wear". The solution is a new seat cover and a redesigned foam pad that the wire can't wear through.

Here is what I get to look at every time I get into my car, and this is the kind of thing that can only get worse.

According to Hinds' service department, the ESP specifically excludes uphostery, and therefore even with this being a known issue, they can't cover it.

Again, redesign to fix means the design was flawed to begin with. Please Ford, fix this!

Rattle Noise Under Vehicle - Heat Shield

TSB 07-11-7

I can't reproduce it "on demand" but it is very disturbing when it happens. Typically during sub-freezing weather. It sounds like I have a gremlin with a jack-hammer under the car.

Cold Start Tip-in Hesitation

TSB 05-26-20

Again, I can't reproduce it on demand, but it definitely happens. Press the accelerator, and the engine skips a beat before revving up. I originally thought it was just characteristic of this particular power-train. Apparently it only happens on very early Fusions, built before 12/9/05 (like mine), and there's a fix for it. I want the fix.

Front Floor Mat Wear

TSB 06-18-14

Here's one that's been known for a long time. "Premature front driver side floor mat wear" according to the text of the bulletin. In fact, Ford did replace these for me once already. The fact that the replacements wore through just as fast means that the issue wasn't actually fixed at the time of my first replacement. Come on Ford, this is just floor mats! One would think these should have been mastered for decades...

That's It

Thank you for your time and attention. We now return you to your regularly scheduled SharePoint Blog.

In this two part series, I show you how to take advantage of SharePoint calculated columns in your SharePoint Designer workflows. Part 1 introduced calculated columns. Part 2 will describe several string manipulation functions, and show you how to consolidate these calculated columns into a "function library" of sorts.

Hide the Clutter

In part 1, I showed you how to use a calculated column to enhance a SharePoint Designer workflow. This is all well and good, but what if you have interim calculations? What if you need to do this manipulation on values entered by your user in the workflow initiation form?  Or maybe you have other types of information you don't want getting in the way of your users when they view the list.

This is where that "Function Library" I talked about comes into play. Technically, in SharePoint terms, it is actually a function list. We will create a list in SharePoint that contains not just the calculated columns, but "input" columns for the functions as well. We will then hide the list in SharePoint designer. Your workflows can still access it, but your users won't even know it is there.

Holiday for Strings

String manipulation is a great application for calculated columns.

One common workflow scenario is processing the items sent to an email-enabled list. Typically, you will want to parse the subject line to find some routing information. This information may take several forms, including:

• The beginning of the subject up to a delimiter
• The end of the subject following a delimiter
• The information contained within a pair of delimiters
• Everything except what is contained between a pair of delimiters

Of course, there are other possibilities, but these should be enough to illustrate the concept.

Building the List

We're going to create a list that supports all of the calculations described in the previous section. To start, of course, we need to create the list. (Remember, SharePoint Designer 2007 doesn't have the ability to create or edit the columns in a MOSS 2007/WSS 3.0 list, so we'll do this in the web UI.)

1. From Site Actions, select "Create"
2. From the Create page, select "Custom List"
3. Give it the name TextFormulas. Since we're ultimately going to hid the list, don't show it on the QuickLaunch
   
4. Click Create

Now that we have a basic list, we need to create our input columns:

1. Display the newly-created TextFormulas list.
2. From the Settings toolbar, select List Settings
   
3. There is a "Title" column created by default. Click on it, change the name to "SourceString", and click OK.
   Note: This step isn't technically required, but it helps things make sense in the Workflow Designer
   
4. Click Create Column
5. Enter "Delimiter1" for the Column Name.
6. Make sure "Single line of text" is selected as the field type, and click OK.
   
7. Repeat steps 4-6, except use "Delimiter2" as the column name.

Once you have your input columns defined, you might think it is time to create your calculated columns. While you could, there is one more step you might want to perform. Because the output from string calculations isn't always obvious, it can be helpful to have some sample data to work with so you can see if the formulas are doing what you want them to.

Add an item to your list:

This item will give you all of the possibilities that we might want to work with - text before, after, within, and outside of, delimiters.

Now we're ready to create the calculated columns.

For the first column we just want the text to the left of the first delimiter. This has the simplest formula, but it isn't necessarily as obvious as you might think. The formula is "trim(left([SourceString],find([Delimiter1],[SourceString])-1))"

We're using three string functions: Trim(), Left(), and Find(). They're also "nested", meaning that we're calling one function from within another function.

The obvious function is "Left()". It takes two parameters, the string we want the left hand side of, and how much of the string we want. Unfortunately, since this is delimited rather than a fixed position, we need to "Find()" the position of the delimiter in the source string.

That's all well and good, but why are we then subtracting 1? That's because we would otherwise return the delimiter itself in our results. Find returns the position of the delimiter, and Left function expects a count of returned characters. Since we don't want the delimiter, we subtract 1 so that we get the position immediately before it.

The Trim() function gets rid of any leading and trailing spaces, as even if you don't see them, these can make comparisons fail.

Create your calculated columns in accordance with this table:

Notice that each formula builds on the one before it - we're using the results of some calculated columns as input to others. This helps us avoid the 8-deep function nesting limit, as well as the 1000 character formula length limit. It also makes them a lot easier to read!

Assuming the formulas are entered correctly, viewing your sample data item will give you these results:

While the value for "AfterDelimiter1" doesn't look very useful in and of itself, that is only because of the particular SourceString we are using. In this case, it is simply an interim value for deriving the BetweenDelimiters result. With a different SourceString it could be the final answer you are looking for. For example, you may have a simple two-part source with a single delimiter, such as "G131131|Memory Failure". That string, with just the pipe (|) as Delimiter1, will result in "Memory Failure" for the AfterDelimiter1 value.

Back to SharePoint Designer

Once you have your function list created, you can use it in your SharePoint Designer workflows.

For this example, we're going to create a workflow on an email-enabled Time Off list. When a new item arrives, we want to parse the subject line to get the reason for the absence and assign it to the reason field. The reason text follows a dash (-), so we're just going to use the "AfterDelimiter1" calculated value.

Note: You may need to enable email-based workflows with the following command before using this example: "stsadm -o setproperty -pn declarativeworkflowautostartonemailenabled -pv true" See this KB Article for details.

When you define your workflow, first check the "Automatically start this workflow when a new item is created box, as shown below:

Click Next.

From the Actions menu, select Create List Item. (Since the menu is built from recently used actions, you may need to select it from the "More Actions" dialog.) You will get a new line in the Actions block that looks like the one below. Click the "this list" link.

Select the TextFormulas list from the dropdown:

The SourceString (*) field will already be chosen, as it is a required field. (This will be the "Title" field if you didn't rename it earlier.)

Click the "Modify..." button. You want to use the E-Mail Subject field from the current item as the source.

After you click OK, Click Add in the Create New List Item dialog, and add the Delimiter1 field. Enter the dash as the Value. When you click OK, the Create dialog should look like this:

Click OK. Notice SPD automatically generates a variable called "Create" as the output of this function. That variable will hold the item ID for the formula item we create. This will be needed later.

A Pregnant Pause

The next action may seem a little odd. We're going to select the "Pause for Duration" action, and then set it to 0 days, 0 hours, and 0 minutes. The reason for this is that the calculations in our function list item don't take place instantaneously. If we were to try to grab the result as the next step, all we would get is an empty string. By telling our workflow to Pause, we give SharePoint a chance to catch up.

Note: Even though we set everything in the pause to zero, the workflow will wait until the next event cycle to continue. This delay may be a minute or so.

Back to Work

Once the workflow comes back from its coffee break, we need to get our value back.

1. From the Actions menu, select Set Field in Current Item.
2. Click the "field" link, and select Reason from the dropdown.
3. Click the "value" link, then click the fx button that appears.
4. Select TextFormulas for the Source, with the Field of AfterDelimiter1.

Now we need to tell the workflow how to find the row we want. Here's where that "create" variable comes into play.

1. Select the TextFormulas:ID field from the Field dropdown
2. Click the fx button beside the Value field.
3. Select Workflow Data for the Source, and Varable: create for the Field
4. If your lookup dialog looks like the one below, click OK
   

The reason lookup should now look like the capture below:

Click OK.

Cleaning Up

Now it is time to clean up after ourselves.

Just as "real" SharePoint developers need to keep in mind the need to "dispose" objects they create once they are done with them, we need to delete the formula record we created now that we are done with it. Fortunately, SharePoint provides a "Delete Item" action item for us. Select it from the Actions menu, and click the "this list" link. As you might guess by now, we're going to use the same "create" variable as before to select the item to delete from the TextFormulas list.

Click OK, then click Finish in the Workflow Designer. SharePoint Designer will then validate the workflow and save it.

Covering our Tracks

Finally, as I mentioned way back at the beginning of this article, you may not want your users to readily see your formula list. Especially since the interim information will be hanging around during that pause. Although we suppressed the list from the Quick Launch when we created it, people can still see it from the "View All Site Content" link. To make it go away from there, we can take advantage of SharePoint Designer's ability to "hide" a list.

To hide a list, first open the Lists folder (you won't need this step for document libraries).

Right-click the list you want to hide and select Properties, as shown below:

Click the "General" tab. Click on the checkbox labeled "Hide from browsers".

Click OK

Now the list will be "invisible" via most normal ways of discovering SharePoint content. That is also why we waited until after the workflow was done in order to hide it. Although you can still see and work with hidden lists in most parts of SharePoint Designer, the function called by the Workflow Designer to enumerate lists is the same one used by the web interface, so you wouldn't be able to select it.

Conclusion

This second part took a bit longer to write than I expected. I hope you found it worth the wait!

In these two articles, I have showed you the power of SharePoint calculated columns, and how to use them in a SharePoint Designer workflow. In the process, I have introduced a number of other concepts, including:

• String functions
• Hidden lists
• Email enabling lists
• Using the output of one calculated column to feed another.
• Workflow variables

Even so, I have barely scratched the surface. I encourage you to explore further the capabilities of calculated columns and SharePoint Designer workflows.