Content and Code Blog

SharePoint 2010 – Here we come!

noreply@blogger.com (Content and Code) — Fri, 05 Mar 2010 20:45:00 +0000

Glyn Clough, Solutions Consultant

We’ve just heard about the official launch date of Microsoft SharePoint 2010, along with the other Office 2010 applications. To businesses, SharePoint will be available from May 12, 2010. To general consumers it will be available in June.

We’ve been working with the new version of the SharePoint product since the Technical Preview stage through to the current Release Candidate stage and are fortunate enough to have been working on several fantastic Proof of Concept projects with some of our clients already. We’re delighted that we now have a firm date in the public domain so that we can plan with our clients how we are going to make the most out of this new release and see all the great new features in action.

Hyphens in domain names get trimmed from account names in SharePoint 2010 User Profile Import

noreply@blogger.com (Content and Code) — Tue, 19 Jan 2010 10:13:00 +0000

Tristan Watkins, Infrastructure Consultant
We've identified that the user profile import in the SharePoint 2010 public beta can't handle hyphens in domain names. The import will succeed but the portion of the domain name preceding the hyphen will get trimmed. When a user logs on a new profile is created but it is orphaned from the imported data. In principal we've been able to work around this by migrating the user profiles with STSADM (thanks to my colleague Martin Hatch for the suggestion) but we haven't put this approach to the test over a sufficient period of time to be able to recommend it firmly yet. We also don't have a mechanism for triggering the update for newly-imported users but it shouldn't be rocket science to come up with a solution to that problem for the duration of the beta.

Microsoft have confirmed this is a problem in the SharePoint 2010 public beta and that a fix will be included in the next release. Their response was on a closed beta forum, so I can't include that detail here, but this is my description from MSDN:

We’ve been trying to launch an internal deployment of SharePoint 2010 MySites but we’ve run in to a problem with the user profile import. We’ve reviewed all of the relevant guidance and have been able to complete an import successfully, but all of the characters before a hyphen in our domain name are getting truncated from the imported username. To illustrate:
PREFIX-BLAH\tristan.watkins becomes BLAH\tristan.watkins after it is imported. When I log in, a new user profile is created for PREFIX-BLAH\tristan.watkins and all of the imported profile data is useless, since it is associated with a non-existent BLAH\tristan.watkins.
I’m assuming this is an issue with the beta version of Forefront Identity Manager 2010 that shipped with SharePoint 2010 beta, but I can't find any information suggesting this is a known issue and for all I know it may actually be a SharePoint issue rather than a problem with FIM. For what it's worth, this was not a problem in SharePoint 2007.

We'll be putting this to the test ASAP whenever the next release is available. I'll update here if we find anything else out or if we find any similar problems in future.
[Also posted on Tristan's personal blog]

Understanding SharePoint Application Security and Elevating Privileges

noreply@blogger.com (Content and Code) — Fri, 18 Dec 2009 16:43:00 +0000

Martin Hatch, Solutions Architect

This post was prompted because of a particularly challenging bit of security that I needed to traverse. I needed some way of presenting the status of a Content Deployment Job (configured in Central Administration) in the Web Application that it relates to.

Seems pretty straight forward?
Well, its not, and this article will hopefully explain why.

RunWithElevatedPrivileges and Application Pool Accounts
So the first thing I looked at was using the good old SPSecurity.RunWithElevatedPrivileges method. This is a well known (and on occasion heavily used) practice for getting around security in SharePoint. But does everyone understand exactly what it does?

In a nut-shell, this method simply changes the currently impersonated user from the currently logged in user to an account called "SharePoint\System" (a.k.a. "System Account").

This account doesn't actually exist, and anyone inspecting the WindowsIdentity or SPUser object in any great detail will spot that this account doesn't actually have a valid SID (Security Identifier). This is because it represents a placeholder.. a flag in SharePoint that tells it to impersonate the Application Pool Account instead of the currently logged in user.

The Application Pool Account has full SharePoint permissions to the Web Application (effectively making it a Site Collection Administrator in every single Site Collection).

So what does this actually mean?

SQL Server Permissions
Believe it or not, SQL Server permissions in SharePoint are extremely simple.

Taking the 3 core databases for each SharePoint Farm:

1. Farm Configuration Database
This contains the core configuration information (servers, URLs, accounts) for the entire SharePoint Farm.

The Setup Account has DBOwner permissions.
All application pools accounts are added to a Database Role called WSS_Content_Application_Pools which has severely locked down read privileges.

2. Central Administration Content Database
This is effectively the content database for the Central Administration site. This contains the SPSite / SPWeb / SPList objects that store all of the content related settings (including Content Deployment Jobs).

Again, the Setup Account (which incidentally will be running the Central Administration Application Pool!) has DBOwner permissions.

All application pools accounts are added to a Database Role called WSS_Content_Application_Pools which has severely locked down read privileges.

3. Web Application Content Database
This is the database (or multiple databases) that contain the Site Collection content for the Web Application.

Here the Application Pool Account (for that specific Web Application) is granted DBOwner permissions. No other accounts are specified!

That is pretty much it. From a security (and "least privileged" perspective) it's a very robust setup. If your application pool is compromised then the application pool account only has SQL permissions to it's own content database.

According to best practice, every Web Application should have it's own application pool account, which again makes sense according to the model above, limiting the surface area for any attack (as one web application being compromised would not have any impact on the other application pools).

This should also make it obvious why you should never make an Application Pool Account a Local or Farm Administrator! You are essentially breaking the security model if you do this (and massively widening the exposed area of your system if that account is ever exposed!).

NTLM authentication and "Double Hop"
The first thing that should scream at you here is that none of the SharePoint user accounts have ANY permissions in SQL. Every single SQL query is executed within a SharePoint Web Application using the Application Pool account!

The reason for this is clear once you understand the limitations of NTLM authentication.

Basically, when you log in to a SharePoint web site, you authenticate with the Web Server (IIS). There is no way for IIS to pass through credentials back to SQL Server because NTLM only supports "single hop" authentication (i.e. from one single machine - the browser - to another machine - the web server). For "double-hop" you need a more robust authentication method such as Kerberos (i.e. from one machine - the browser - hop to another machine - the web server - hop a second time to a third machine - the database server?).

Note - This is why you need Kerberos to use pass-through authentication with 3rd party systems (such as CRM or other LOB systems).

That’s all great .. but what do I care?
Well, this all nails down to where the object is that you are trying to access, what the SQL permissions are on that object.

Lets take the example of accessing a Content Deployment Job.

The first problem you will hit is that your account needs to be a Farm Administrator. We already know that making the Application Pool an admin account is bad for security.

So as an alternative you could use ASP.Net Impersonation to get around the SharePoint API, but as we discussed above, this doesn't solve the NTLM "single-hop" problem (your query is still going to execute in SQL using the Application Pool account, regardless of which account you are impersonating!)

Using .Net Reflector (tsk!) tells us that the Content Deployment Job information is stored in an SPList in the Central Administration Content Database. Using RunWithElevatedPrivileges simply executes using the Application Pool account (which we know from the SQL Permissions above, has very limited permissions).

So lets assume you tried to use Impersonation ... what happens?

Well, you get a nasty "Exception in HRESULT" error message.
Delving in to the SharePoint Diagnostics Logs tells something like " does not have EXECUTE permissions on 'proc_EnumLists' in ".

Basically running that code tries to execute a Stored Procedure in SQL in the Central Admin database which the Application Pool Account doesn't have access to! Your code managed to fool the SharePoint API into thinking you have permissions, but good old SQL Server stops you short (just as it should ... good server!)

So what can I do?
Well, the first thing to note is that you won't always run into this problem.
Many of the Farm level options (including access SSP and User Profile properties) can be gotten around in other ways, but when something like the above happens, your options are limited to 3 potential solutions:

Ignore all of the best practice. Make your application pool account an administrator, and spend your days hiding from the network security admins and hoping it doesn't all go wrong.
Create a dedicated Web Service, which executes as an admin account. Use this to farm out your "privileged" code, and make sure you lock it down tight as a drum so you can't get to it from outside of the SharePoint farm!
Don't do it .. and tell your users that it was a stupid idea in the first place!

Now I admit, Options 1 and 3 probably won't go down too well, and Option 2 is the best option but still has it's issues (running a Web Service as an admin account is still a security risk, if a smaller one than running the entire public facing Application Pool as an admin account!)

Summary
We ended up opting for Option 2, admittedly locking it down so that the URL was never published and it would only accept connections from other servers in the farm (so that end users could never access it).

Hopefully you now have a better grasp of SharePoint Application Security, what that super-method "SPSecurity.RunWithElevatedPrivileges" is actually doing and why it doesn't always work!

Comments and feedback welcome! :)

[Also posted on Martin’s personal blog]

Building a SharePoint 2007/2010 development environment - Part VI: Issues and Results

noreply@blogger.com (Content and Code) — Wed, 02 Dec 2009 22:17:00 +0000

Tristan Watkins, Infrastructure Consultant
In the first five parts of this series I covered the project objectives and the system design, then turned my attention to the Hyper-V host image build, automated deployment and the guest virtual machine build. In this post I review some of the questions and issues we've encountered after a few months of working this way and some overall reflections on the approach.

Issues

Guest user accounts

Guest virtual machines have been configured in a workgroup in order to conserve resources that would be spent on domain services. Additionally, developing on a domain controller is less than ideal for a number of reasons including performance tuning, administrative complexity, start-up times and security.

I created the development virtual machine with 160 local user accounts that have been logged on to the portal and the MySite application in order to create a basic profile. If there is a need to script creation of local user accounts, the net user command will be useful. However, this will be of limited assistance for complex profile requirements, since there is no way to synchronise with a directory and since the local users have no associated profile data, but it may be helpful for testing or demonstration.

If LDAP user accounts or other directory objects are required for development purposes (user profiles for instance), consider using Active Directory Lightweight Directory Services. This is the successor to Active Directory Application Mode (ADAM) in Windows Server 2003. It is a Windows LDAP directory that supports user and group objects without a full-blown domain infrastructure.
There will be some scenarios when a full domain services infrastructure is required for development. In those cases it may be preferable to run a second virtual machine as a domain controller.

Hibernate and Sleep

Hibernate and Sleep are disabled automatically when Hyper-V is installed. This is by design. Hyper-V disables this functionality, as the guest virtual machines could be damaged by a Hibernate or Sleep operation in the host if they were not saved gracefully. If, on the other hand, all virtual machines had to be put in to a saved state before a host machine could be put to sleep or hibernated, this would mean extending the wait time for these operations to unacceptable levels, as they are also automatically triggered by low battery warnings. Unfortunately we need to live with this behaviour.

Do not travel with a running laptop

Putting a running laptop in a bag will cause it to overheat quickly and is likely to damage hardware.

Improvements to Start-up and Shutdown times

These builds should start up and shut down in less than two minutes (closer to 90 seconds). Keep in mind that virtual machines can be safely saved and work can be resumed quickly when the machine is restarted. Since all of the development work will be taking place inside the virtual machine, this should reduce the Hibernate/Sleep annoyance.

Virtual PC won’t run on Windows Server 2008 R2

Windows Virtual PC will not work on Windows Server 2008 R2, as it was designed specifically for Windows 7. Earlier versions of Virtual PC may install on Windows Server 2008 R2, but they will not co-exist with Hyper-V, so do not install them.

Hyper-V role won’t work after SysPrep

This shouldn’t be an issue, as we have set up automated deployment, but it’s worth noting that this is a known issue. There are time-consuming work-arounds to fix some of the problems that this will cause, but they are best considered as a last resort.

Colour management

Colours are limited to 16-bit in Hyper-V guests. If a fuller spectrum is required, it should be possible to test in full colour in a browser on the host.

Resolving host names from an internal domain

During our pilot we identified that fully-qualified domain names resolved successfully but host names would not resolve without the full domain name. To satisfy this requirement we have added our internal DNS suffixes to the ICS Connection inside the development virtual machine.

Manually adding DNS suffixes

If a network adapter in a guest virtual machine loses these settings by deletion/re-creation of the adapter, or for some other reason, the setting can be re-entered as follows:

Go to the IPv4 properties on the ICS Connection and select Advanced.
On the DNS tab select the Append these DNS suffixes (in order) radio button.
- Add internal.domainname.local and other.domainname.com.
Un-tick the DNS registration box.
Select OK, OK and Close.
Make sure that this change is captured in all snapshots as necessary.

Internet Explorer (64-bit version)

Adobe flash player does not currently support 64-bit browsers. You'll have to use the 32-bit IE or another browser if you want to view flash files. We recommend using the 32-bit version by default.

Hyper-V Manager UAC prompt work-around

If the UAC prompt on Hyper-V Manager is annoying, try launching Server Manager and navigating to Hyper-V in the Roles node. This has the added benefit of exposing the Hyper-V event log messages and service states in that top Hyper-V node. These are not visible in Hyper-V Manager. Awareness of these messages and the service statuses will help to resolve Hyper-V issues faster.

Test DVD burning

In our pilot we identified that the DVD burner drivers don’t work for burning in Windows Server 2008 R2 on a Dell XPS M1330. This was also true on a Lenovo laptop. Chipset updates, driver updates and a Microsoft KB registry hack all failed to make a difference. The Matshita (Panasonic subsidiary) site does not support the products directly (they point to the laptop manufacturer). Dell and Lenovo had not released new drivers when we launched. As DVD burning has changed in Windows Server 2008 R2 this may have a wider impact.

Bluetooth doesn’t work

The Bluetooth stack is missing from Windows Server 2008 and Windows Server 2008 R2. In Windows Server 2008 there were fairly elaborate means of porting the stack from Vista, but results appear to be spotty at best.

WorkItemTypeDeniedOrNotExistException when trying to open work items

This error occurred in the first release of our guest build because I installed Visual Studio 2008 SP1 before the Team Foundation Client (TFC), so the TFC did not get upgraded. The fix is to un/re-install Visual Studio 2008 SP1, or to make sure that the TFC is installed before Visual Studio 2008 SP1.

NUMA nodes and RAM allocation

It is important to not exceed NUMA node limits when assigning RAM to virtual machines, although this will not apply to many laptops, as most will have an SMP architecture. It is beyond the scope of this post to go in to NUMA nodes in great detail (and in truth, my understanding of it does not reach beyond a few hours of research), but the limits in your environment should be understood so that performance does not suffer. As a starting point it's worth confirming the type of CPU architecture and looking at this in more detail if it is NUMA. The performance and capacity requirements for Hyper-V document on TechNet explains this well:

Configure the correct amount of memory for Hyper-V guests. During the testing, no change had a greater impact on performance than modifying the amount of RAM allocated to an individual Hyper-V image. Because memory configuration is hardware-specific, you need to test and optimize memory configuration for the hardware you use for Hyper-V.

The initial goal of the testing was to make the Hyper-V image as similar as possible to the physical hardware image against which it was being compared. Based on that goal, the Hyper-V images were originally allocated 32 gigabytes (GB) of RAM, which was the same amount of RAM as was on the physical servers being tested. However, the initial test results showed that with that configuration, the Hyper-V images could sustain a load that was only about 70 percent of the load on the physical hardware. After investigating the Event Viewer on the Hyper-V host machine in the Windows Server 2008 Custom Views, Server Roles, Hyper-V Events, it was discovered that the RAM for the Hyper-V images was being spread across multiple non-uniform memory access NUMA nodes. This information confirmed that performance declined when memory was allocated across nodes. After trying different configurations it was determined that for the hardware being used, 8 GB of RAM was the maximum that could be allocated to a Hyper-V image without crossing NUMA nodes.

To reiterate, this means that in Microsoft's tests, Hyper-V performed significantly worse with 32GB allocated to a virtual machine than it did with an 8GB allocation. The exact size of the NUMA node boundary will vary by vendor, so make certain to gain an understanding of the number of nodes in your system. Divide the total RAM by the number of nodes in order to find the memory limits of a virtual machine. This does not mean that additional virtual machines can’t be run beyond a NUMA node boundary, if there is sufficient RAM available. The node boundary is the limit of optimal process performance. Beyond this limit, the virtual machine will suffer from degraded performance because it needs to use memory from an alocal address space.

However, NUMA isn't the only thing to worry about when finding an optimal RAM allocation. Based on test results during our pilot we could push our virtual machines to up to 2250MB RAM, depending on the amount of activity in the host. In some cases it may be possible to get up to 2500MB RAM for a virtual machine on a 4GB RAM system, but this was not consistently achievable in our tests. If it’s necessary to achieve that, the virtual machines should be started up soon after booting and before any major client application activity is started on the host machine. Client application activity should be kept to a minimum when allocating this much RAM to virtual machines. We also found that host performance was often reduced to an intolerable level whenever there was less than 2GB RAM available to the host for an extended period of time. 1.75GB RAM may be achievable, but this should be tested extensively for your needs.
Additionally, saving a virtual machine's state becomes risky when there is less than 2GB available to the host, as the machine will not resume from the saved state if there is insufficient resource available to it.

Periodic but routine loss of connectivity on the host machine

As I've been tracking here, we've documented repeat problems with periodic (but routine) loss of connectivity on the host machine. This is still an open issue. More info here:
Routine loss of connectivity on a Hyper-V host’s external connection
More on routine loss of external network connectivity on Hyper-V hosts (not guests)

Hyper-V performance suffers during graphics-intensive operations

This has been covered by Ben Armstrong in considerable detail and I'm continuing to track it:
Hyper-V graphics performance and SharePoint 2010 development
Hyper-V graphics performance is on the way… if you need a new laptop
The definitive word on Hyper-V high-end graphics performance

Aero Glass

Unfortunately, due to the graphics performance issues in Hyper-V mentioned above, there is a significant graphics performance hit when using Aero Glass. This does not slow down overall systems performance, but graphics-heavy operations will suffer in most Hyper-V environments. To this end, we do not recommend installing Aero Glass, but if you want to put it to the test feel free.

How to enable Aero Glass

Make sure the Desktop Experience is activated
- On the Dell XPS M1330, make sure BIOS A14 or later is installed
  - Confirm the latest NVIDIA drivers for Windows 7 x64 are installed
  - Turn on the Desktop Window Manager Session Manager service and switch to automatic start
  - Turn on the Themes service and switch to automatic start
  - Switch to an Aero theme

Sidebar

It is also possible to add the Windows 7 Sidebar to the host dekstop. We haven't tested this extensively enough to provide documentation on the best approach, but we have done it successfully. If there is sufficient interest in this technique I will add a follow-up post in future.

Storage

Be prepared for snapshots to increase the local storage requirements considerably. Some of our developers are legitimately struggling to work on three projects concurrently with 300GB local storage. One option we are considering is eSATA over PCMCIA as a means of increasing total spindle speed and storage but we have yet to begin testing this approach. We are specifically interested in eSATA since Hyper-V does not support system VHDs on USB. If we pursue this option I'll post the results.

Results

Putting things in perspective

It's worth repeating that we're asking this system to do many things it was not intended to do. It is a server operating system with an enterprise virtualisation technology. The Microsoft virtualisation team will tell you that Hyper-V was not designed with developers in mind. To quote Ben Armstrong:

As is being discussed at length here - Hyper-V does not play well with high-end video cards (which are far more common on desktops than servers). Hyper-V also disables sleep and hibernate, as well as increasing the power utilization of the computer. All of these things would need to be addressed before we could even consider putting Hyper-V in a desktop product.

In short, we bent this system to development needs because of the strength of the technology, despite these imperfections. There are fundamental compromises that can't be avoided when using a server operating system as a mobile workstation but we believe that we can deliver SharePoint projects as a team better with this technology than without it.

Hardware

Whether laptops are the ultimate hardware solution is a different can of worms, which I've chosen to avoid in this series of posts. I've tailored the approach to laptops since that is what we have and the approach can be ported to workstations or shared virtual infrastructure.

The developer experience and the bottom line

There's no question that using snapshots, import and export in Hyper-V adds a complex tier to the development experience and there will be a learning curve for those who are less familiar with virtualisation or don't use the advanced features often. However, we have achieved an immediate and measurable gain in stability and environment consistency through the use of standard builds, snapshots and exported project-defined environments.

Conversely, it's worth keeping in mind that as desirable as standardisation is, there are times when it hinders more than it helps and on those occasions a non-standard build may be more appropriate. Considering alternative builds is a much less cumbersome proposition with the combination of WDS, Shrink Volume and Dual-booted systems or the new Native boot from VHD. The key consideration to keep in mind is that most other approaches will entail a sacrifice of what I lump together as the "management benefits" of Hyper-V (snapshot, import and export). For instance, you may consider allowing a team to develop on native operating systems for a project, but then a team member may lose a day if they need to rebuild their system, or the support team may need two days to build an environment in Hyper-V later on, or a team member may need to split her time with a team who use Hyper-V for their project, or the original project may fork. Memories of project difficulties gone-by come flooding back. While it's always worth considering options, if you spend time identifying a standard approach for your business, it's probably best to stick with it unless there's a truly compelling reason not to.

[Also posted on Tristan's personal blog]

Building a SharePoint 2007/2010 development environment - Part V: Guest Build

noreply@blogger.com (Content and Code) — Fri, 27 Nov 2009 01:31:00 +0000

Tristan Watkins, Infrastructure Consultant
In the first four parts of this series I covered the project objectives and the system design, then turned my attention to the Hyper-V host image build and automated deployment. In this post I describe a SharePoint 2007 virtual machine build.

Where’s the SharePoint 2010 build?

In short, we're working on it. I've produced a new SharePoint 2010 beta virtual machine for this environment but we're not yet ready to publish build guidance. Stay tuned. Additionally...

SharePoint 2010 memory requirements

We have validated this environment for use with SharePoint 2010 virtual machines, but performance will be inadequate for development unless 4GB RAM can be dedicated to the SharePoint 2010 VM. It will run with 2GB but it will be too slow to do anything beyond lightweight demonstration. Also, be sure to check if your system has a NUMA CPU architecture. If it does, only allocate memory within NUMA node boundaries, as performance degrades when accessing memory from alocal nodes. See the performance and capacity requirements for virtualising SharePoint guidance for an introduction to the issue, but be aware that NUMA architecture is less common in laptops so it may not be an issue.

Virtual networking recap

As explained in part two of this series, we have created three virtual networks on the Hyper-V host, although this development virtual machine will only have two network adapters. The first will be "plugged in" to the Hyper-V ICS Network. The second is "plugged in" to the Hyper-V Internal Network. We plug the Hyper-V External Network in for testing purposes only. For more information about how I've designed these networks please refer to the design post.

Working with virtual networks

The guest virtual machine's "ICS Network connection" will receive the host’s LAN or VPN connectivity (and an IP address via DHCP) from the host's adapter on the Hyper-V ICS Network. The second adapter is “always on”, in that communication between the host and the guest will be completely separated from the host's provision of ICS to the guest. This second adapter has a fixed IP address on a network that's dedicated to providing communication between guest VMs and a resilient, permanent connection in the host. This enables:

Persistent network sharing.
RDP connections to guest VMs.
HOSTS file entries and BackConnectionHostNames security.
Multiple IP addresses on static adapters in guests, for multiple sites with SSL, etc.

Static IP address allocation on the ICS connection in the guest is not a supported configuration and results in instability.

System Build

Installation

Create a new virtual machine in the location of your choice. After the virtual machine has been created, modify these settings:

Assign multiple processors to the virtual machine (for most laptops you will want to assign all of them).
Assign sufficient RAM, leaving at least 2GB RAM for the host machine (1.75GB bare minimum - host performance is too slow with only 1.5GB available).
Connect the Hyper-V ICS Internal Network to the first NIC.
Connect the Hyper-V Internal Network to the second NIC.
Point the DVD drive at an ISO file or installation media for Windows Server 2008.
- It's also possible to network-boot, following a similar approach to the host deployment from WDS. To network-boot a Hyper-V machine, add a legacy network adapter (it must be legacy) and modify the BIOS settings to boot from the network.
- We could have installed Windows Server 2008 R2 as we did on the host, but at the time of our pilot it was still only in Release Candidate, so we opted to stick with a tested OS in order to compare the development experience with our current environments. I built our SharePoint 2010 Technical Preview and Beta environments on Windows Server 2008 R2 RTM without problem.

Turn on the virtual machine and install Windows Server 2008 Standard. On completion, log in for the first time, insert Hyper-V Integration Services from the Virtual Machine Connection's Action menu and reboot when prompted.

Initial configuration

These steps establish basic connectivity and improve usability:

Log in to the new virtual machine and enable remote desktop connections using any version (in order to support older versions of Royal TS). If all versions of Remote Desktop are current then select the secure option.
Test that the ICS connection is able to retrieve an IP address and that the connection is receiving connectivity from the host machine. If there are any problems connecting to the LAN or web, make sure that the connection is shared in the host and that the ICS connection in the guest is on DHCP. Also check the firewall settings. No settings should need to be modified but make updates if necessary.
- Temporarily plug the Hyper-V External network in to this adapter for testing if needed. This is a good way of identifying if a problem relates to ICS itself or if it's a guest firewall issue.
Add DNS suffixes to the ICS connection if you want to be able to resolve host names on your LAN (without having to use the fully-qualified domain name). ICS will not pass through primary or connection specific DNS suffixes.
- Go to the ICS Connection’s IPv4 properties.
- Select Advanced.
- On the DNS tab select the Append these DNS suffixes (in order) radio button.
- Add domain suffixes for any named resources that should be resolved using host headers.
Configure IP settings on the Internal Connection in the guest.
- Configure IP addresses as 192.168.200.100.
- Set the Subnet Mask as 255.255.255.0.
- A default gateway and DNS settings are not necessary.
Remove IPv6 from all connections in the host and guest. See earlier guidance for more information.
Give all adapters sensible names.
- I chose "ICS Connection" and "Internal Connection".
Test browsing to \\192.168.200.1 to test connectivity over the Internal network.
Optionally, turn off IE Enhanced Security Configuration (ESC) for administrators.

Patching

Apply Windows Server 2008 SP2.
- Reboot.
Patch current.
- Reboot if prompted and patch again if patches are available. Reboot again if needed and repeat until current.

Shutdown and snapshot

Shut down the virtual machine and take a snapshot. Rename the snapshot with an appropriate name, such as “Windows installation and patching complete”. You will be able to roll back to this state if you have problems with the Windows configuration.

Adding Prerequisites

Adding Web Server (IIS) role

Start up the virtual machine and add these Windows Role services:

Web Server
- Common HTTP Features
  - Static Content
  - Default Document
  - Directory Browsing
  - HTTP Errors
  - HTTP Redirection
- Application Development
  - ASP.NET
  - .NET Extensibility
  - ISAPI Extensions
  - ISAPI Filters
- Health and Diagnostics
  - Logging Tools
- Security
  - Basic Authentication
  - Windows Authentication
  - Request Filtering
- Performance
  - Static Content Compression
  - Dynamic Content Compression
Management Tools
- IIS Management Console
- IIS Management Scripts and Tools
- Management Service
Windows Process Activation Service
- Process Model
- .NET Environment
- Configuration APIs

Adding Features

Add these Windows Features:

Desktop Experience
Optional: Windows System Resource Manager (and Windows Internal Database)

Installation of the Desktop Experience will prompt for reboot. It is not necessary to do so immediately. Note that when the reboot happens, Windows will restart a few times. This is a normal part of the Feature configuration.

Additional preparation

HOSTS file entries

Created the following HOSTS file entries for the web applications:

192.168.200.100 SSPAdmin

192.168.200.100 MySite

192.168.200.100 Portal

DisableLoopbackCheck or BackConnectionHostNames updates

Either disable the Loopback Check or add BackConnectionHostNames entries. Since ICS prevents inbound connectivity to these virtual machines we feel comfortable disabling the loopback check, as this would be a considerable annoyance to developers otherwise. For unisolated systems I would always recommend putting the BackConnectionHostNames entries in place. Spencer Harbar has summarised the background information best.
To disable the Loopback Check, add a new DWORD DisableLoopbackCheck value “1” to HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Lsa

Creating the Source repository in the host and sharing to guests

To understand these steps it is worth reviewing the Local source code storage on the host machine section of the design post.

Create a new user account in the guest machine called SourceShare, with same password as the account that was created by the WAIK unattend file in part four of this series. This password should be strong, as the share will be visible to authenticated users on the host machine.
Confirm that the CreateSourceShare.bat script from part four has created a folder on the host system that is shared it to the SourceShare user with full control. Remove “Everyone” from the sharing permissions and replace with “Authenticated Users”.
In the guest virtual machine, map the Z drive to \\192.168.200.1\Source, specifying the SourceShare account credentials and that it should be reconnected at login. Save the password.
Test the connection by opening the Z drive. If there are problems opening the drive, browse to \\192.168.200.1\ and troubleshoot from there.
If users have trouble with the strength of the password, allow them to change it to something that they will remember, but make certain that it is changed in both the host and the guest and make sure they know that the change will need to be applied to all snapshot states and any other virtual machines that use this repository in the host.

Note: if you would prefer to store source code in the virtual machine it is not necessary to follow these steps. The benefits of preserving code outside of the snapshot state need to be weighed up against the drawbacks of rolling back to a state when some of that code did not exist, and users of the system need to be aware of the implications of using either approach. Clearly this approach adds some complexity and any approach should be validated for your needs.

Create local user accounts for services and application pool identities

SQL service accounts (which should be for obvious purposes):
- SQLSvc
- SQLAgent
- SQLAnalysis
- SQLReporting
SharePoint service accounts and application pool identities:
- MossSetup (installation account)
- CentralAdmin (farm account)
- SSPAdmin (Shared Services Administration site application pool identity)
- MySite (MySite application pool identity)
- Portal (Portal application pool identity)
- SearchService (MOSS Search service account)
- SharedServices (the SSP web service identity)
- ContentAccess (the crawl account)

Assign the MossSetup account local administrative rights.

Activate windows

Add a valid Windows license and activate the system. Aside from obvious compliance issues, if this is not done you will see repeated 12321 Security-Licensing-SLC “Token-based Activation failed” warnings in the event logs.

Shutdown and snapshot

Shut down the virtual machine and take a snapshot. Rename the snapshot with an appropriate name, such as “Windows configuration complete”. You will be able to roll back to this state if you have problems with the SQL installation.

SQL 2008 Setup

Installation

Start up the virtual machine and run the SQL 2008 installer. Select all features.

Give the instance a name or choose the default instance name according to preference.
Set up SQL services under these accounts:
- SQLSvc
- SQLAgent
- SQLAnalysis
- SQLReporting
- Everything else can run as Network Service.
Disable Analysis, Reporting and Integration services. We do not run these services by default in order to optimise performance. However, we do install the services to speed up deployment as needed. Use as necessary in your environment.
Select Windows Authentication.
Select the local Administrator account as a Server Administrator.
Install Reporting Services in SharePoint integrated mode.

After installation completes, launch the SQL Management Studio and assign the MossSetup account DBCreator and SecAdmin rights. It will not be necessary to assign other permissions directly in SQL, as the setup account will assign DBCreator/SecAdmin rights to the farm account.

Patching

Add the SQL Server 2008 Feature Pack and SQL Server 2008 SP1 and. Reboot as necessary and patch with the latest Cumulative Update.

Shutdown and snapshot

Shut down the virtual machine and take a snapshot. Rename the snapshot with an appropriate name, such as “SQL 2008 configuration complete”. You will be able to roll back to this state if you have problems with the MOSS installation.

MOSS deployment

Installation and initial configuration

Start up the virtual machine. Log in with the MossSetup account and follow these steps.

Run the MOSS installer with (at least) SP1 slipstreamed. Select the Complete Server Type.
When the installer completes, un-tick the “Launch SharePoint Products and Technologies” wizard tick box.
Open a command window, running as Administrator and navigate to the BIN in the 12 Hive. Run PSCONFIG –cmd configdb –create to manually specify friendly names for the Central Admin databases and to assign the CentralAdmin farm account as the user. Specify both the Central Admin configuration and Content databases or they will receive an awful GUID for a name. Find more information about PSCONFIG on TechNet.
Launch the SharePoint Products and Technologies Configuration wizard from the start menu. Retain the connection to the newly-created farm databases and proceed through the configuration wizard according to your build requirements. I recommend that you specify a standard Central Administration port if you use one and authenticate using NTLM.
Once the wizard completes, launch Central Administration to confirm that the site is up and running.
There are tips and tricks that you may want to include for your developers, for instance Andrew Connell’s suggestions and an interesting use of junction points from the Share Notes blog.
- Consider putting a desktop, task bar or start menu shortcut to the 12 hive.
- Add an environment variable for the 12 hive’s BIN.
  - Go to the machine’s Advanced System Properties (right-click My Computer and select Properties).
  - Click the Environment Variables button.
  - In the System variables pane, scroll down to Path and select “Edit”.
  - Clicking in the Variable Value will put the cursor at the end of the line. Type a semi-colon.
  - Copy and paste the path to the 12 hive’s BIN, typically:
  - C:\Program Files\Common Files\Microsoft Shared\web server extensions\12\

Shutdown and snapshot

Shut down the virtual machine and take a snapshot. Rename the snapshot with an appropriate name, such as “MOSS installation complete”. You will be able to roll back to this state if you have problems with the farm configuration.

Farm build and configuration

Start up the virtual machine and launch Central Administration from the Start menu or navigate to it in Internet Explorer.

Enable Intranet Settings for the Central Admin site. This option should appear as a prompt directly beneath the tabs.
Grant Launch and Activation permission to the local WSS_WPG and WSS_ADMIN_WPG groups in the IIS WAMREG DCOM component. This prevents system error 10016. See KB920783 for more information.
- If necessary, do the same thing on the oSearch DCOM componet, per KB953137.
Enable the MOSS Search service, running under the <MACHINE NAME>\SearchService account with “Reduced” performance, using all WFE servers.
- If the account name is entered without the <MACHINE NAME> prefix, you will likely see this error:
  “An unhandled exception occurred in the user interface.Exception Information: OSearch (SearchService)”.
We have not configured any other central administration settings in order to provide the farm in the cleanest state possible. That said, perform any additional configuration here as needed, but be aware that many settings are not suitable for all projects.

Create the Shared Services Provider

Configure the farm’s shared services as prompted by the left navigation. This will step through the creation of the SSP administration and MySite web applications and then the Shared Services web service itself.

Create a new SSP administration web application when prompted.
- Use the host name and load balanced URL that was set up above: http://SSPAdmin
- Use the application pool identities as created above: SSPAdmin
- Select NTLM authentication, unless there is a reason not to.
- Give the content database for the web application an appropriate name, such as WSS_CONTENT_SSPAdmin, according to internal naming conventions.
Create a new MySites web application when prompted.
- Use the host name and load balanced URL that was set up above: http://MySite
- Use the application pool identities as created above: MySite
- Select NTLM authentication, unless there is a reason not to.
- Give the content database for the web application an appropriate name, such as WSS_CONTENT_MySite, according to internal naming conventions.
Proceed with the SSP configuration:
- Use NTLM authentication.
- Specify the SharedServices application pool identity.
- Do not use SSL, as this traffic is not leaving the server.
- Typically in a stand-alone devlopment environment it should be OK to use the default SSP database naming suggestions, but adhere to internal standards where they apply.
Wait for the configuration wizard to complete. This can take some time.
Visit the SSP administration site to confirm that the web application has been successfully created.
In Central Administration, navigate to Self-Service Site Management and enabled Self-site creation for the MySite web application.
Visit the MySite link in Central Administration to confirm that the web application has been successfully created.
Log in to the SSP Administration site and take these steps:
- Specify a default content access account: ContentAccess
- Grant full personalisation services rights to the local administration account.
  - This account can be used to grant rights to yet-to-be-created local users of the system once they receive this build, if desired.

Create the web portal

In Central Administration create a new web application.

Use the host name and load balanced URL that was set up above: http://Portal
Use the application pool identities as created above: Portal
Select NTLM authentication, unless there is a reason not to.
Give the content database for the web application an appropriate name, such as WSS_CONTENT_Portal, according to internal naming conventions.
Create a root site collection using the collaboration portal template. Make the local administrator account and the Moss Setup accounts site collection administrators – to be changed by local users on receipt of the build.
Visit http://portal to confirm that the site collection has been successfully created.

Shutdown and snapshot

Shut down the virtual machine and take a snapshot. Rename the snapshot with an appropriate name, such as “MOSS configuration complete”. You will be able to roll back to this state if you have problems with the patching.

Patch current

Start up the virtual machine and patch the environment to whatever state is needed. We typically recommend patching current. During the course of our pilot we used the April 2009 Cumulative Update. When we released the base virtual machine to our entire development team we launched with the June 2009 Cumulative Update. Once patched current, test that all sites are still working as expected.
Full patching guidance is available on the Update Center for Microsoft Office, Office Servers, and Related Products site.

Shutdown and snapshot

Shut down the virtual machine and take a snapshot. Rename the snapshot with an appropriate name, such as “June 2009 CU applied”. You will be able to roll back to this state if you have problems with developer tool installation.

Tool installation and export

Developer Tool installation

Start up the virtual machine and install the common development tools for your organisation. These are the standard tools used by our developers. There are also some licensed products that are installed for some users outside of the scope of the base build.

Microsoft Office SharePoint Designer 2007 SP2
Microsoft Visual Studio Team System 2008 Developer Edition (we created a separate snapshot for users that need Visual Studio 2005 rather than running them concurrently)
Microsoft Visual Studio Team System 2008 Team Explorer
Microsoft Visual Studio Team System 2008 Team Foundation Server MSSCCI Provider
TFS Power Tools
Microsoft Visual Studio Team System 2008 Developer Edition SP1
Microsoft Visual SourceSafe
WSPBuilder Extensions 1.0.5
StyleCop
Fiddler2
Sandcastle
Sandcastle Help File Builder
HTML Tidy
PDF X-Change PDF Viewer
Firefox with Firebug
Safari
Chrome
Opera
SharePoint Manager 2007
Microsoft Silverlight
Adobe Flash for IE
Adobe Flash for Firefox, Safari and Opera
Microsoft Forefront Client
Test Mail Server

On completion of installation, defragment the disk and wait for completion.

Shutdown and snapshot

Shut down the virtual machine and take a snapshot. Rename the snapshot with an appropriate name, such as “Developer tools installed”. You will be able to roll back to this state if you have problems with the farm configuration.

Snapshot branches

If you will be using different versions of Visual Studio (or anything else) it’s worth creating distinct branches for each version. You could adopt an approach like this:

Install all of the independent tools, shut down and take a snapshot, renamed to something like "Developer tools except VS installed".
Install Visual Studio 2008 and patches. Defragment, shut down and take a snapshot, renaming to something like "Visual Studio 2008 installed and patched current".
Apply the "Developer tools except VS installed" snapshot. Install Visual Studio 2005 and patches. Defragment, shut down and take a snapshot, renaming to something like "Visual Studio 2005 installed and patched current".
The two Visual Studio snapshot are now siblings - children of the "Developer tools except VS installed" snapshot node. A third sibling could be created in this same manner for Visual Studio 2010 Beta 2 (for instance).

We install Visual Studio last in the installation sequence in order to maintain consistency among these variants as deep in to the snapshot tree as possible. To give another example, I've just built a SharePoint 2010 Beta environment with Visual Studio 2010 Beta 2. In that environment there was only one instance of Visual Studio but two SharePoint branches - one for SharePoint Foundation and another for SharePoint Server 2010. In that tree I installed all developer tools and Visual Studio before installing either SharePoint edition. If you have similar branching needs, plan to install those items last and structure your snapshot trees accordingly.

Export

You may choose to export the entire virtual machine rather than the final snapshot, depending on how standard you want the final build to be. If a developer can role back to an earlier state then the standardisation of the build is at risk. In our case, we’ve exported the final snapshot (or each of the final sibling nodes in the snapshot tree, since some users use Visual Studio 2005). This state is vanilla enough that it can be used as a common starting point for a wide range of projects. It has even been used as a starting point for projects that extend beyond SharePoint to accessibility solutions, Commerce Server and BizTalk.

This is not to say that this build fits all sizes, but it does fit most of them while obtaining the benefits outlined in the first post in this series. Some of the limitations of this approach are discussed in the next part of this series.

[Also posted on Tristan's personal blog]

Building Accessible SharePoint Systems - SharePoint User Group UK

noreply@blogger.com (Content and Code) — Thu, 26 Nov 2009 12:56:00 +0000

Glyn Clough, Solutions Consultant

Last night it was good to see Martin Hatch presenting at the SharePoint User Group in London. Martin is a Solution Architect and a colleague of mine at Content and Code who has worked extensively on building accessible SharePoint solutions - most recently for the Royal National Institute of Blind People.

Key highlights for me were the use of Rendering Templates and Control Adapters to override out of the box controls and how they are displayed and function. These techniques can be used on both MOSS 2007 and SharePoint 2010, and are not solely aimed at delivering accessibility improvements.

Martin has now made available the materials from his presentation:

If you would like to read more about Martin’s work with the RNIB and the SharePoint Accessibility Solution (SAS) from Content and Code please see the accessibility section of our website.

[Also posted on Glyn’s personal blog]

Building a SharePoint 2007/2010 development environment – Part IV: Automated deployment

noreply@blogger.com (Content and Code) — Wed, 25 Nov 2009 14:43:00 +0000

Tristan Watkins, Infrastructure Consultant
In the first three parts of this series I covered the project objectives and the system design, then turned my attention to the Hyper-V host image build. In this section I will look at automating deployment of that host operating system. This is lengthy, but there's a lot to cover.
Before I begin, I need to reiterate a "gotcha" that I stumbled across during the pilot. If you're thinking of testing this out by adding the Windows Deployment Services role to a machine that's using Internet Connection Sharing (ICS), think again. They don't play nicely together. Rather, you'll be able to deploy the physical machines but any ICS recipient network adapters will be left without connectivity. If you only have one physical machine for testing, consider deploying WDS to a new virtual machine or adding the role within an existing virtual machine.

Imaging

To capture the host OS image we need Windows Deployment Services (WDS), which is a Server Role in Windows Server 2008 R2 (and Windows Server 2008, but we won't be bothered with earlier incarnations of WDS, RIS or ADS). To specify configuration options during deployment we will attach an unattend file to the captured image. We'll create it using the WAIK for Windows 7, which includes Windows System Image Manager (Windows SIM).

Preparation

Before we begin, note that WDS needs to be deployed in an Active Directory domain (as a member or a DC) with DHCP and DNS.

To install WDS, add the Server Role through the Role Summary pane in Server Manager

Once launched, configuration options can be selected for WDS, such as default naming patterns and default unattend files. You will not be able to commit any configuration changes unless they are made by a Domain Administrator. If this is a test environment and you are unable to get Domain Admin rights, WDS can still be used with a bit more of a hands-on approach (manually naming servers and joining to the domain, etc.). These configuration options are covered in more detail below

To download the Windows Automated Installation Kit for Windows 7 (WAIK), visit microsoft.com and grab the latest version

Mount the ISO file and install Deployment Imaging Servicing and Management (DISM), which includes the Windows System Image Manager (SIM)

A good ISO mounting tool is Magic ISO Maker. Daemon Tools jumped the shark

To use WAIK, you will need to have access to Windows Server 2008 R2 installation media

If the server is a virtual machine, consider attaching ISO files of the installation media as optical drives on the IDE controllers in the virtual machine's settings, as this may make things easier

I have created two additional optical drives in my WDS virtual machine, with a Windows 7 ISO file mounted in one virtual optical drive and a Windows Server 2008 R2 ISO file mounted in the other. This allows me to maintain unattend files for both operating systems without having to mess about with real optical/removable media

Ensure that both the WDS machine and the soon-to-be-captured Hyper-V host are connected to the network
If testing WDS on a live network, be certain that there are no other PXE boot servers on the environment before adding the WDS role, as this may result in network installation chaos otherwise

Configure WDS

This guidance covers some of the settings in WDS that are applicable to this use. I don't touch on all of them. It should go without saying that WDS deployment should be considered in much greater detail than these summary steps and that administrators should familiarise themselves with WDS and WAIK before unleashing them. That said...
To configure WDS, log on to the server as a domain administrator and launch Server Manager. Expand the Windows Deployment Services node in "Roles". Right-click the server that you want to configure and select "Properties".

PXE Response

Choose how strictly you want to control deployment of images. If this is a test environment, configure the role for known and unknown devices. Ideally this configuration is a bit more considered for production. Note: this is distinct from the PXE Boot Policy, which governs the behaviour of the PXE boot process rather than restricting which devices can be booted in to WDS. Think of this as the device setting and PXE Boot Policy as the user setting.

Client Naming Policy

These settings control the auto-generated naming format. For instance, "Dev-%8Username-%02#", breaks down as follows:

“Dev-”
“%8” specifies an eight-character variable
“Username” is the variable, meaning that the first eight characters of the user's name will be used in the computer name
“-” adds a hyphen
“%02#” specifies a two-character auto-incremented number at the end of the string

Computer Account Location

Specify an OU where new instances of the image should be created.

PXE Boot Policy

Configure this setting to preference. It controls the behaviour of the PXE boot once it has been initiated (either from a one-time boot menu or BIOS boot order change). Once in the PXE boot a DHCP lease is acquired, followed by the behaviour defined here. PXE can either abort and continue with the next BIOS boot option, automatically proceed or prompt the user to press F12 to confirm PXE boot. Think of this as the user setting and PXE Response as the device setting. If unsure what to choose, prompting is probably the best option to choose.

Default Boot Image and Default Unattend file

To simplify things we attached an unattend file directly to the captured image, as spelled out in John Howard's, slipstreamed Hyper-V article. The unattend file doesn't exist yet, as we haven't delved into WAIK yet. For now it's sufficient to know that these WDS configuration settings determine distinct boot images and default unattend files for different processor architectures. Since we're attaching the unattend file to the image directly we don't need to worry about providing values for these settings.
It's worth reviewing the overlapping options in WAIK and WDS for these defaults settings, as it's far from straight-forward. TechNet covers this beyond the scope of this article.

Joining a Domain

This setting allows for the newly deployed machines to be automatically joined to the domain.

Network settings

Specify DHCP, Multicast, Transfer Settings (throttling), domain discovery and the UDP port range. Default settings should suffice in a test environment. Approach these choices with care if this environment contains any valuable resources beyond these test machines.

Prepare the Capture Image in WDS

The Capture Image is a PXE-booted version of WinPE specifically designed for capturing custom install images. When PXE booting to capture an image, make sure to select this image rather than a boot image.

Mount an ISO or insert install media for Windows Server 2008 R2

As mentioned above, it might be easiest to add the ISO as a DVD drive on the VM's IDE Controller. This will also be useful for WAIK configuration later

Add the Boot WIM to the Boot Images directory

Right-click Boot Images in WDS and select "Add Boot Image"
Make sure to select the right CPU architecture for the boot image

Note: if you forget to do this, you can capture an x64 OS with an x86 Capture Image, but it may be slower to do so. Also, an x64 Capture Image will not be bootable unless the machine has an x64 processor

Create a Capture Image from the Boot Image

Right-click the newly added boot image in WDS and select "Create Capture Image"
This will launch a wizard with a link to a help file if the wizard is unclear
This is necessary for creating a custom install image in WDS

Generalise the host build

We're finally ready to get back to the host build. Note: if the host has been built as a virtual machine (as recommended at the top of part III of this series), take a snapshot of the virtual machine before running SysPrep so that the reseal count is not incremented for future updates to the build.
To generalise the host build, run sysprep /oobe /generalize /shutdown. This can also be done from the GUI by double-clicking sysprep.exe.

Capture the install image

Start the host machine and invoke the one-time boot menu (commonly F12).

Select PXE, NIC or network boot (terminology varies)

Depending on the configuration of the Windows Deployment Services (WDS) server it may be necessary to press F12 again to confirm PXE boot

If this isn’t done, the machine will boot from hard disk and the machine will need to be SysPreped again

Select the Capture Image that was created above

Again, remember to select the correct processor architecture

In the Windows Deployment Services Image Capture wizard, choose a location to save the new image

This location can be the same disk that's being captured. I suggest creating a folder called D:\Captured Image so that the directory can be removed with the post-deployment script below

Note: the drive letter will not be "C:", as that's the WinPE Capture Image that we're "in" at this point. The System disk will probably appear as "D:"

Allow time for the image to be captured locally and then for it to be transfered to WDS

In my experience, the captured image does not appear in WDS immediately. I've needed to re-launch Server Manager to view it and in some cases a reboot was necessary
If for some reason the network transfer of the image to WDS fails, it can be manually copied to the WDS server and added, but make sure to reboot the server as described above before taking this step or the images may be duplicated and this could get confusing

Creating the Unattend file

Open the Windows System Image Manager from the Start > All Programs > Microsoft Windows AIK menu.

In the Distribution Share pane, select or create a distribution share in the location of your choice. I created mine in D:\WAIK\DistributionShare
In the Windows Image pane, choose Select Windows Image and choose the Windows Server 2008 R2 \sources\install.wim file from install media. Select the Standard Full edition when prompted
Paste the XML below in to a text file and save it as DevBuild.XML
In the Answer File pane, open DevBuild.XML below as a starting point for your selections

DevBuild.XML

<?xml version="1.0" encoding="utf-8"?>
<unattend xmlns="urn:schemas-microsoft-com:unattend">
<servicing>
<package action="configure">
<assemblyIdentity name="Microsoft-Windows-Foundation-Package" version="6.1.7600.16385" processorArchitecture="amd64" publicKeyToken="31bf3856ad364e35" language="" />
<selection name="Microsoft-Hyper-V" state="true" />
<selection name="VmHostAgent" state="true" />
<selection name="Microsoft-Hyper-V-Management-Clients" state="true" />
<selection name="WirelessNetworking" state="true" />
<selection name="CoreFileServer" state="true" />
<selection name="SearchEngine-Server-Package" state="true" />
</package>
</servicing>
<settings pass="windowsPE">
<component name="Microsoft-Windows-International-Core-WinPE" processorArchitecture="amd64" publicKeyToken="31bf3856ad364e35" language="neutral" versionScope="nonSxS" xmlns:wcm="http://schemas.microsoft.com/WMIConfig/2002/State" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
<SetupUILanguage>
<UILanguage>en-GB</UILanguage>
</SetupUILanguage>
<InputLocale>en-GB</InputLocale>
<SystemLocale>en-GB</SystemLocale>
<UILanguage>en-GB</UILanguage>
<UserLocale>en-GB</UserLocale>
</component>
<component name="Microsoft-Windows-Setup" processorArchitecture="amd64" publicKeyToken="31bf3856ad364e35" language="neutral" versionScope="nonSxS" xmlns:wcm="http://schemas.microsoft.com/WMIConfig/2002/State" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
<DiskConfiguration>
<WillShowUI>OnError</WillShowUI>
</DiskConfiguration>
<Display>
<ColorDepth>32</ColorDepth>
<HorizontalResolution>1280</HorizontalResolution>
<RefreshRate>60</RefreshRate>
<VerticalResolution>800</VerticalResolution>
</Display>
<ImageInstall>
<OSImage>
<InstallToAvailablePartition>true</InstallToAvailablePartition>
<WillShowUI>OnError</WillShowUI>
</OSImage>
</ImageInstall>
<UserData>
<ProductKey>
<WillShowUI>OnError</WillShowUI>
<Key>XXXXX-XXXXX-XXXXX-XXXXX-XXXXX</Key>
</ProductKey>
</UserData>
</component>
</settings>
<settings pass="specialize">
<component name="Microsoft-Windows-IE-ESC" processorArchitecture="amd64" publicKeyToken="31bf3856ad364e35" language="neutral" versionScope="nonSxS" xmlns:wcm="http://schemas.microsoft.com/WMIConfig/2002/State" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
<IEHardenAdmin>false</IEHardenAdmin>
<IEHardenUser>true</IEHardenUser>
</component>
<component name="Microsoft-Windows-IE-InternetExplorer" processorArchitecture="amd64" publicKeyToken="31bf3856ad364e35" language="neutral" versionScope="nonSxS" xmlns:wcm="http://schemas.microsoft.com/WMIConfig/2002/State" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
<CompanyName>Your Company</CompanyName>
<DisableAccelerators>false</DisableAccelerators>
<Home_Page>http://yourhomepage.com</Home_Page>
<IEWelcomeMsg>true</IEWelcomeMsg>
<SuggestedSitesEnabled>false</SuggestedSitesEnabled>
</component>
</settings>
<settings pass="oobeSystem">
<component name="Microsoft-Windows-Deployment" processorArchitecture="amd64" publicKeyToken="31bf3856ad364e35" language="neutral" versionScope="nonSxS" xmlns:wcm="http://schemas.microsoft.com/WMIConfig/2002/State" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
<Reseal>
<ForceShutdownNow>false</ForceShutdownNow>
</Reseal>
</component>
<component name="Microsoft-Windows-Shell-Setup" processorArchitecture="amd64" publicKeyToken="31bf3856ad364e35" language="neutral" versionScope="nonSxS" xmlns:wcm="http://schemas.microsoft.com/WMIConfig/2002/State" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
<OOBE>
<HideEULAPage>true</HideEULAPage>
<ProtectYourPC>3</ProtectYourPC>
</OOBE>
<StartPanelLinks>
<Link0>C:\Program Files (x86)\Mozilla Firefox\firefox.exe</Link0>
<Link2>C:\Program Files (x86)\Safari\Safari.exe</Link2>
<Link1>C:\Program Files (x86)\Opera 10 Beta\opera.exe</Link1>
</StartPanelLinks>
<TaskbarLinks>
<Link0>%windir%\system32\notepad.exe</Link0>
<Link1>C:\Program Files (x86)\code4ward\Royal TS\</Link1>
<Link2>C:\Program Files (x86)\Internet Explorer\iexplore.exe</Link2>
</TaskbarLinks>
<UserAccounts>
<AdministratorPassword>
<Value>encrypted</Value>
<PlainText>false</PlainText>
</AdministratorPassword>
<LocalAccounts>
<LocalAccount wcm:action="add">
<Password>
 <Value>encrypted</Value>
<PlainText>false</PlainText>
</Password>
<DisplayName>SourceShare</DisplayName>
<Name>SourceShare</Name>
</LocalAccount>
</LocalAccounts>
</UserAccounts>
<TimeZone>GMT Standard Time</TimeZone>
</component>
</settings>
<cpi:offlineImage cpi:source="wim:z:/sources/install.wim#Windows Server 2008 R2 SERVERSTANDARD" xmlns:cpi="urn:schemas-microsoft-com:cpi" />
</unattend>

Make changes to this file as needed for your environment and save the modified file. I highly recommend reviewing these selections for suitability in your environment. Some sections like the license key and passwords (bolded above) have been replaced with nonsense values. It's also worth reviewing the options that I haven't selected. The Windows SIM Help documentation is excellent and it should be a quick learning curve.

Attach the Unattend file to the image in WDS

In the WDS Install Images node, expand the relevant Image Group, find the captured image and right-click to select the Properties pop-up. At the bottom of the General tab there's a tick box for allowing the image to install in unattended mode and a file selection button for picking the unattend file created above. This will apply the Unattend settings to our captured, sysprep'd image when it's deployed.

Create the Post-deployment PowerShell configuration script

Before I begin this section, I have to disclaim that my scripting skills are fairly remedial. That said, these scripts have done the trick for us and have saved our support team a good deal of time reconfiguring environments manually.

These steps are necessary because only certain things can be specified in the System Image Manager. For instance, it is possible to configure network adapters and it is possible to add Hyper-V but it is not possible to add Hyper-V, create Hyper-V networks and then configure IP properties for the host's new network adapters on those Hyper-V networks. To this end, the unattend file above specifies only the addition of the Hyper-V role and this script automates the next two steps. Additionally, this script will create the Source files share on the host machine (as discussed in part II of this series) and remove the local "Captured Image" directory that was created when WDS captured the image to the local drive.

Create a folder called "Automate" or similar
Visit the PowerShell Management Library for Hyper-V Codeplex page and download HyperV.zip

run the installer. The script below needs to run from the same directory as hyperv.format.ps1xml and hyperv.ps1

In the "Automate" folder create a batch script called CreateSourceShare.bat, as follows:

c:
MKDIR C:\Source
NET SHARE Source=C:\Source /GRANT:"Authenticated Users",FULL
ICACLS C:\Source /GRANT SourceShare:F
RMDIR "c:\Captured Image"

Create a PowerShell script called HyperVNICAndShareScript.ps1 with these contents:

netsh interface set interface name="Local Area Connection" newname="Virtual Network Switch"
. .\hyperV.ps1
New-VMExternalSwitch "Hyper-V External Network" -ext "Broadcom"
Start-Sleep -s 5
New-VMInternalSwitch "Hyper-V Internal Network"
Start-Sleep -s 5
New-VMInternalSwitch "Hyper-V ICS Network"
Start-Sleep -s 5
netsh interface set interface name="Local Area Connection" newname="External Network Connection"
netsh interface set interface name="Local Area Connection 3" newname="Internal Network Connection"
netsh interface set interface name="Local Area Connection 4" newname="ICS Network Connection"
netsh interface ipv4 set address name="Internal Network Connection" source=static address=192.168.200.1 mask=255.255.255.0
C:\Automate\CreateSourceShare.bat

Deployment

At long last we are ready to test deployment of the captured image to a target machine. Start the host machine and invoke the one-time boot menu (commonly F12).

Select PXE, NIC or network boot (terminology varies)

Depending on the configuration of the Windows Deployment Services (WDS) server it may be necessary to press F12 again to confirm PXE boot

Select the x64 boot image for Windows Server 2008 R2
The WinPE installation process will be nearly identical to installation from media - just make sure to select the captured image when prompted
Once the installation has completed, run PowerShell as Administrator and set the execution policy to allow remote-signed scripts. For more information see Technet:

Set-ExecutionPolicy RemoteSigned

Press "Y" to accept the policy change
Run HyperVNICAndShareScript.ps1
Confirm that settings match the values assigned in the scripts

Manual post-deployment configuration tasks

Various settings

It's possible that some of these settings can be automated, but we opted to configure them manually rather than spending time testing the automation of them. They will all be revisited in a later phase.

Make sure that the BIOS has Hardware Assisted Virtualisation and Data Execution Prevention enabled
Turn on Write-caching and turn off Write-cache Buffer Flushing on the physical disk's hardware policy
Set Hyper-V default locations as desired. I have chosen:

C:\Hyper-V\Virtual Hard Disks

C:\Hyper-V\Virtual Machines
Disable IPv6 on the network adapters. SharePoint in Hyper-V guidance suggests this is a good idea, as I've commented on before
Set up Internet Connection Sharing from the host's external network connection to the host's ICS network connection. By default we share HTTP, HTTPS and RDP

Drivers

We reduced our WDS/WAIK testing time by skipping driver installation automation. As a work-around we are capturing the host image with driver installer files included so that they can be quickly installed post-deployment. We will be introducing automated deployment in a future phase.

Role-based Authorisation and local administration rights

It may also be desirable or necessary to configure role-based authorisation for Hyper-V. John Howard has produced excellent guidance on the model in five parts. This is a link to the fifth. If this is not a requirement then don't forget to make the recipient of the build a local administrator, as this will be required if role-based authorisation is not configured.

Import virtual machines

As a part of a routine deployment the base virtual machine would be imported, booted up and tested before it's handed over to the recipient, but since we've not built that machine yet, I'm skipping that step for now. The virtual machine build will follow in part five of this series.

[Also posted on Tristan's personal blog]

Great SharePoint 2010 Learning Resource

noreply@blogger.com (Content and Code) — Tue, 24 Nov 2009 17:53:00 +0000

Glyn Clough, Solutions Consultant

I was recently shown a great resource for SharePoint 2010 on Channel 9, Microsoft’s developer focussed video content site. Not being a dev, I’ve previously not spent a lot of time looking around Channel 9, but that’s all about to change…

There’s some fantastic video content and presentations on 2010, which will serve as a great introduction for both developers and non-developers alike. The content is collated in a course called ‘SharePoint 2010 Developer’ and grouped into 15 sections, so you can easily focus on the content you need.

- SharePoint 2010 Developer Home Page

It’s fantastic that Microsoft and the guys who’ve helped create/produce/test this content have put so much effort so early on into the product lifecycle on educate all the users and practitioners of SharePoint out there. We’re a million miles away from where we were with MOSS! Great work.

[Also posted on Glyn’s personal blog]

Working with SharePoint 2010: The “Developer Consultant”

noreply@blogger.com (Content and Code) — Thu, 12 Nov 2009 09:25:00 +0000

Tobias Lekman, Technical Architect

From all the new features in SharePoint 2010, most seem to be focused around developers and development productivity. There are lots of talk about the new Visual Studio 2010 tools for SharePoint, the “Business Connectivity Services” and sandboxed web parts.

Does this mean that we will spend more or less time writing actual code?

At the SharePoint Conference in Las Vegas, I sat in on the “best practises” session where few new topics were covered. 90% of the material was taken from existing SharePoint guidance documentation and sandboxed solutions were touched upon. However, one single line stuck in my mind - “developers, know your platform”.

I have time after time seen developer-heavy implementations of solutions that could have been created using configuration of existing components in SharePoint. At the same time, I have seen solutions where one or two features of SharePoint have been used (and abused) to encompass all scenarios. This is simply because the implementation has been done by someone who has limited knowledge about SharePoint as a platform.
From a developer’s point of view, here are the major changes in SharePoint 2010:

Less code and more configuration
Less hand-hacking of definition files and more exports of prototypes into solution packages
Less custom solutions and more reliance on out-of-the-box solutions combined with empowered end users

As an example: I recently had very similar requests from two very different clients. They both wanted “powerful” reporting across data sources and against multiple criteria. It also needed to be easily reconfigurable to fit future needs. Both clients mentioned analysis and data mining in loose terms around “dimensions”. This could easily have been steered into a long and expensive project with OLAP data warehouses, reporting models, Excel Services and Performance Point Server, and the developer in me started talking about this approach on both occasions. However, when I demonstrated what could be achieved by connecting Excel and Access directly to the SharePoint list data and performing pivot reports against the data, both clients were more than happy. In the end, I ended up proposing more time on end user training and configuration and hardly any custom development.

The same story can now be applied to requirements surrounding accessibility, standards compliance, integration of line-of-business systems and external data and workflows.

SharePoint 2010 does empower the developer, but more importantly – it makes the development cycles shorter, provides better tools for packaging prototypes into solutions and further empowers end users to interact with the platform with rich client tools.

Out: Large development teams. In: Developer consultants!

[Also posted on Tobias' personal blog]

Building a SharePoint 2007/2010 development environment – Part III: Host image build and performance benchmarks

noreply@blogger.com (Content and Code) — Mon, 09 Nov 2009 09:42:00 +0000

Tristan Watkins, Infrastructure Consultant

Having agreed the project objectives and designed the system, I turned my attention to the Hyper-V host image build. This is a high-level build guide with start-up time and baseline memory consumption benchmarks at key milestones. These benchmark figures were taken from the Windows Server 2008 R2 Release Candidate build and are admittedly a bit imprecise. However, they do provide an overall indication of system performance as things were added to and removed from the installation. Although I do not have precise figures on RTM improvements, I spot-checked a few of these benchmarks when I rebuilt the system on RTM. Start-up times improved slightly at each milestone. In fact, the final benchmarks came in at 100MB less idle memory used in the RTM release.

Capturing a virtual image
For the pilot and the benchmark figures listed below, the image was built on hardware and captured with Windows Deployment Services (WDS). Since then, we've rebuilt the system as a Hyper-V guest and we're capturing the image by PXE booting the SysPrep'd virtual machine from a legacy network adapter. We take a snapshot before SysPrep as a rollback point for future updates to the image. Virtual PC Guy explains this approach in more detail in his geeking out with WDS post post, which I've linked here previously. I'll also be discussing the Windows Server 2008 R2 WDS build and the new version of WAIK in more detail in the next post in this series.

System specification

Dell XPS M1330
- Intel Core 2 Duo T8100 2.1 GHz
- 4.00 GB 667 MHz DDR2 SDRAM
- 7200 RPM 320GB SATA HDD
- nVidia GeForce 8400M GS graphics card
- Windows Server 2008 R2 Build 7100 (RC)

Host build
Initial OS optimisations
I installed the system from a USB pen drive (I had to try this given how easy it is now). On first login I started optimising the system as follows:

Write-caching is enabled in the physical hard disk hardware policy
- Buffer flushing is turned off
- We are comfortable with this risk since laptops run from battery or with battery backup at all times
IPv6 is turned off on host network adapters per SharePoint's Hyper-V performance and capacity guidance
- This is optional, and should not be turned off if you have IPv6 requirements
IE Enhanced Security Configuration (ESC) is turned off for administrators
- Again, depending on security requirements, this may not be acceptable
The following services are disabled (all optional):
- Certificate Propagation
- Desktop Window Manager Session Manager
  - Leave this enabled if using Aero Glass
  - In our pilot experience, we identified that Aero Glass suffers from Hyper-V graphics performance degradation enough that it worsens the overall experience
- IP Helper
  - Leave this enabled if using IPv6
- Remote Registry
- Windows Remote Management (WS-Management)

For more information on Windows service hardening, see Black Viper's site. This list of services could possibly be extended, but at the time of testing service hardening information for the new operating system was harder to come by than it is presently. It should be noted that the Windows Server model of turning services on if needed means that a new system is already considerably harder than a new Windows 7 system.

Next I booted the system, logged in and let it rest for at least fifteen minutes to capture the idle memory benchmark.

Benchmarks
Memory benchmark: as low as 525 MB idle.
Start up time to logon: 37 seconds.
Time to desktop: 52 seconds.

BIOS settings
I installed BIOS A15 (includes alleged performance improvements) and made the following BIOS modifications:

NIC settings = Enabled with PXE (required for image capture)
HDD Acoustic Mode = Performance
Virtualization = Enabled
Internal Cellular = Off
Wireless Switch = Wi-Fi + BT

Note: if you have internal cellular and use it, don't turn it off in the BIOS and adjust the last setting accordingly. Also note, BIOS A14+ is required if exceeding 4GB RAM.

Next I booted the system, logged in and let it rest for at least fifteen minutes to capture the idle memory benchmark.

Benchmarks
Memory benchmark: as low as 525 MB idle.
Start up time to logon: 37 seconds.
Time to desktop: 48 seconds.

Adding drivers and features
Drivers

SigmaTel Audio driver
UPEK driver for TouchChip fingerprint Coprocessor (WBF advanced mode)
nVidia GeForce 8400M GS driver
- Note: on one occasion this driver install forced a reboot despite requesting that it be postponed - be careful
Intel and Ricoh chipset drivers (Intel may not be necessary)
There is still one device without a driver:
- BCM2045 (Bluetooth)
- This is because Windows Server 2008+ does not have the Bluetooth stack

Features

Enabled the desktop experience and WLAN featurs
- Requires .NET Framework 3.5 and Ink and Handwriting support

Next I rebooted the system to complete the driver installations. On reboot, I set the Windows Audio service to Automatic and started it up. I tested Audio and the wireless network adapter.

Install Windows Search
I chose not to index any drives. This is primarily installed so that Outlook Search can be turned on with little hassle if desired

Other OS modifications
Disabled Shutdown Event Tracker, as developers seem to hate it and it isn't really necessary on a laptop

Following these changes I rebooted, logged in again and let the system rest for an extended period so that I could capture the next idle memory benchmark.

Benchmarks
Memory benchmark: as low as 622 MB idle.
Start up time to logon: 50 seconds.
Time to desktop: 1 minute 12 seconds.

Installing Microsoft Office client tools

Office 2007 (defaults, including InfoPath and OneNote)
Visio 2007
Office Communicator 2007 R2
Patched current

Benchmark with Office turned off and Communicator closed
Memory benchmark: as low as 708 MB idle when first launched, idling at closer to 760 MB after a couple of minutes. Proper benchmarks after lengthy idle were not captured here unfortunately. The next set are more useful/reliable.
Start up time to logon: 56 seconds.
Time to desktop: 1 minute 18 seconds.

Installing other client software
I installed the latest versions of all majors browsers, JRE 6, PDF X-Change Reader, Royal TS, browser plugins like Flash/Silverlight and browser debugging tools like Fiddler and Firebug. Some users of the build have specific license requirements that are accommodated manually after the system is deployed.

Benchmarks
Memory benchmark: as low as 740 MB idle.
Start up time to logon: 56 seconds.
Time to desktop: 1 minute 20 seconds.

Summary
That's it! We met out stated goal to keep the build as lightweight and uncluttered as possible in order to improve performance and reduce the support burden of laptop rebuilds. This system is not much more than a hypervisor, office client applications, web debugging tools and browsers. Keep in mind that I have been building an image that will be captured by WDS. A number of settings will be applied with an unattend file and some additional configuration will be taken care of by script, or manually post-deployment.

Acute observers will note that I haven't actually added the Hyper-V role. That's because Hyper-V doesn't get along well with SysPrep. We account for that with WAIK and scripting, as revealed in more detail in my next post.

[Also posted on Tristan's personal blog]

Geographically distributed deployments of SharePoint 2010

noreply@blogger.com (Content and Code) — Fri, 06 Nov 2009 13:50:00 +0000

Tobias Lekman, Technical Architect

SharePoint 2010 brings a large amount of improvements for geographically distributed SharePoint installations.

SharePoint deployments can be categorized in the following segments:

Uni-centric deployments

SharePoint is installed on a single data centre with a geographically dispersed user base.

Multi-centric deployments

SharePoint is installed on multiple data centres across regions/continents and clustered user bases close to the data centres.

Hybrid deployments

SharePoint is installed on multiple data centres with a geographically dispersed user base.

Globally dispersed users can be categorized into the following segments:

Local/LAN users

Users with high bandwidth, 1mb connections and up and low latency at less than 50ms per roundtrip

Continental latency

Users with high bandwidth (ADSL, cable) but higher latency at up to 125ms per roundtrip

Global users

Users with high bandwidth but with even higher latency at up to 300ms per roundtrip

Low bandwidth users

Users in developing countries, satellite links, mobile connections. These users would have low bandwidth and very high latency – too disperse to estimate

Out of these figures we can estimate WAN performance. These figures show degradation in time for the first page to load (PLT1) over different segments assuming that LAN users are the starting point at X seconds:

Segment	PLT1	Example
LAN	X	3 seconds
Continental	2X-4X	6 to 12 seconds
Global	4X-8X	12 to 24 seconds
Low band	??	Very hard to estimate

SharePoint 2007 deployments

The vast majority of SharePoint 2007 deployments used the uni-centric deployment as network related issues were too great to overcome. This is due to the nature of a farm, where all servers communicate with each other as a “single entity”. The farm has a “shared services provider” which is a site on the farm that contains services that are shared between web applications on the farm. These include searching and user profile storage and management.

SharePoint 2007 can be installed over separate databases during a global deployment so that team/department site databases can reside closer to its intended user base, but the shared services and farm configuration database must still be shared. This causes a lot of communication over global connections and keeping these at high speed and low latency is both costly and difficult.

New capabilities in SharePoint 2010

Cross-farm/multiple instance Shared Services

The “shared service provider” is not obsolete. Services are provisioned per web application and can be hosted either on the same farm or on a remote or “master” farm. The services can also be configured in multiple instances (for simultaneous access over load balancing) or customised per web application. Remote shared services, i.e. published services, also involve caching between farm instances for performance optimization. The most common shared services are user profile storage, managed meta data services (farm/cross farm shared content types and taxonomy), search and BDC/BCS connectivity services.

Uninterrupted log shipping

Creates read-only farms by using SQL log shipping of the entire farm. This can be very useful for creating local copies of content for users on remote continents. Log shipping sends a diffs (differential data packets) of the data and is therefore very fast. As an added bonus, you can use the read-only farm as a disaster recovery platform where you can turn off the read-write farm and make the read only farm the master read-write farm.

FSSHTTP

This catchy acronym stands for “File Synchronization via SOAP over HTTP” and works by reading/writing diffs between the SharePoint server and Office 2010. When reading a file, a checksum is sent to the server. If there are any changes, then only the change diff is downloaded to the end user. This will drastically reduce network usage when working with large documents and/or large numbers of documents.

ODC

Another acronym that stands for “Office Document Cache”. This application lives on the client machine and manages diff replication of documents between the client and the server. Normally, when saving a document, the Office application will wait for the document to finish saving. With ODC, the document is saved to the “save queue” and is synchronized in the background. This also allows for offline access of already opened documents as well as "offline saving”. Multi-master merges are solved by the ODC engine.

SharePoint Workspace

This is the new version of “Groove”, an application that looks like any other Office application but contains an explorer-like view of the SharePoint data. You can synchronize documents and lists for offline viewing and editing.

Office Web Applications

Documents can be viewed/previewed and edited directly in the browser using client script and AJAX. Not only does this mean that users do not need the full Office client on their local machine but it uses much less bandwidth.

Mobile Access View

The mobile version of SharePoint has been vastly upgraded and now covers the entire site collection. It can be customized as in the previous version but also includes Office Web Applications. The mobile view can be used with standard browsers as well. Documents can be viewed as text only or as image snapshots. In a demonstration I saw a comparison of a document at 432kb using up only 14kb when viewed in mobile view.

Windows 7 Branch Cache

This works as a content proxy for any asset such as documents and images. Branch caching can work in peer-to-peer mode, where computers in the workgroup are queried for a document before going to the server. You can also install a dedicated branch cache server using Windows 2008 R2 on the network. The branch cache is again diff aware and can greatly increase performance on satellite worker hubs.

[Also posted on Tobias' personal blog]

Building a SharePoint 2007/2010 development environment – Part II: Design

noreply@blogger.com (Content and Code) — Thu, 05 Nov 2009 10:33:00 +0000

Tristan Watkins, Infrastructure Consultant

In the first part of this series, I introduced the pros and cons of various SharePoint development approaches and the objectives of this system redesign. In this part I will focus on design choices and conclusions, starting with the core technology.

Why we’ve chosen Hyper-V
There are broadly five decisive factors: performance, management features (like snapshots), cost, 64-bit OS support and a full host OS (not just a virtualisation administration console):

Hyper-V is now in its second iteration and has proved to be one of the best-performing virtualisation technologies on the market. With laptops we need to take advantage of every performance gain that we can find.
Perhaps most importantly, and most often overlooked, Hyper-V is free if you're already running Windows Server 2008 or Windows Server 2008 R2as a client OS

While I work for a Microsoft Gold Partner and Hyper-V is a key part of Microsoft’s future strategy (full disclosure), I believe that most SharePoint developers will have at least an MSDN subscription and a license for a Windows Server operating system, so I believe this is compelling for most SharePoint development environments

Hyper-V is one of the few virtualisation technologies that supports 64-bit guest operating systems and within that narrow range of choices, it is one of the few that allows a user to log on to a host machine and control virtual machines within it concurrently

For instance, VMware ESXi is also a high-performance Type-1 hypervisor that supports 64-bit operating systems, but there is no host machine other than a virtualisation administration console

It also has associated costs (support, management tools, etc) that we would prefer to avoid

With Windows Server 2008 R2 and Hyper-V, developers can use client tools within a familiar operating system while accessing virtual development environments at the same time

This is also true of VMware Workstation, but it's a Type-2 hypervisor and will have relatively poor performance

We need to have 64-bit OS support for Windows Server 2008 R2 and SharePoint 2010 - both of which do not have 32-bit flavors

Our approach
Workgroup development
By building our virtual machines in a Workgroup, we no longer need to worry about SharePoint installation/(re)configuration difficulties, as we will import the same identical virtual machine on everyone’s Hyper-V host. This would not be possible if the virtual machine was a member of a centralised domain because the domain controller would receive chatter from many identical machines and everything would quickly start to unravel

Alternately, we could run Active Directory Domain Services within a virtual machine, but this has a performance overhead that is best avoided unless it is absolutely necessary. Additionally, developing on a domain controller is not ideal

While developing in a Workgroup presents challenges for profile imports, these are far from insurmountable when LDAP directories like AD LDS or SQL users can substitute in many scenarios. The only evident need is a scenario where Active Directory Domain Services (more than just user accounts) are required, and that's certainly not going to be true of enough projects that it should form a part of the base build. To this end, as requirements for Domain Services are identified we will provide new builds, as they are likely to be very project-specific

Networking
Internet Connection Sharing network
In order to isolate identical virtual machines from each other and from network resources, I've created a Hyper-V internal network dedicated to receiving Internet Connection Sharing (ICS) from one of the host's active network connections. This is an internal network like any other Hyper-V internal network - it just so happens that the host's adapter on this network will be receiving ICS from another of the host's connected adapters. Any connection can share to the ICS network - even if Hyper-V doesn't natively support external networks of that type. Depending on the need, we will share to this ICS network from:

Hyper-V host external connections (by default)
Wireless
Mobile broadband
VPN

ICS also introduces a layer of NAT between the guests and the physical network, preventing inbound connections to guests over these networks. This is desirable as it is how we achieve physical network isolation, and is the reason why we've chosen ICS over Bridging. In the ICS Settings we enable outbound RDP, HTTP and HTTPS connections over ICS by default, although it may be useful to enable other common outbound network protocols like FTP and SMTP. Outbound connectivity from our guest virtual machines is primarily used for connecting to TFS over HTTPS.

Internal network
We have also created a Hyper-V Internal network to support "always on" communication between the host machine and the guest virtual machines. Guest virtual machines can also communicate with each other over this network.

We cannot rely solely on the ICS connection. Guest virtual machine IP addresses on the ICS network will change because they receive them via DHCP from the host's ICS connection. This is just how ICS works. The host's ICS adapter becomes a gateway on 192.168.137.1. Any Hyper-V guests on that network pick up DHCP from the host and are automatically assigned an address on the 192.168.137.xxx (255.255.255.0) IP range.

As we rely on HOSTS file entries for RDP connections from host to guest and for browsing to guest SharePoint sites from the host, we need fixed, reliable IP routing and name resolution, so we use this second network for that purpose. The hosts and guests have both been built with fixed IP addresses on this range.

Because this is a Hyper-V internal network, there is no risk of network collisions on these IP ranges (which are identical for all users). Internal traffic never leaves the machine.

Project builds
By providing self-contained environments, our technical leads and/or architects can now customise the base builds to create project-specific virtual machines that can be exported to all members of a team, reducing system/configuration inconsistencies. In practice, the build lead/architect will export the final snapshot of the project environment, which all team members will use as a starting point for project development. As the project progresses, updated builds can be released in this same manner.

Clean hosts
Host machines will be cleaned of development tools and data, allowing quick provisioning. Development tools will not need to be reconfigured, as they will reside in virtual machines that can be exported/imported to any host. Guest base build virtual machines will be provided with as much SharePoint configuration as possible in order to make them light on reconfiguration, disposable and re-deployable.

Project resumption
Project-specific exports will reduce the need for storage of multiple virtual machines - as much as possible. We will retain one virtual machine export in public storage per-project. This will reduce the amount of time involved with resurrecting environments after project completion.

Optimisation
Host machines are built on Windows Server 2008 R2, optimised as much as possible and reduced to the lightest weight achievable. The build will broadly include:

Hyper-V R2
Microsoft Office applications
All browsers
No development tools (these will all live in guest machines)

The only exception to this is the Team Foundation Client which TFS administrators manually install. We have not been able to pick domain users from within a Workgroup environment

Content provisioning
By deploying a single project-specific virtual machine, we can bake content in to the project build, ensuring consistency and reducing the overhead associated with re-deploying content.

Improved testing and reduced volatility through the use of snapshots
By using snapshots we are able to test code and configuration changes without volatility. The benefits of this technology include:

Capturing restore points at milestones in a server build
Capturing a stable state before attempting volatile configuration changes
Capturing a stable state before testing code changes
Creating an initial restore point after importing a virtual machine and re-configuring network adapters (when required) and making other preferential settings

This saves re-importing and reconfiguring network adapters if a machine needs to be rolled back to its initial state

Exporting a virtual machine to capture a problem when trouble occurs

An exact instance of a problem can be captured and shipped out for support, without having to re-create the problem in a distinct environment. This will only apply tot self-contained environments, however

Local source code storage on the host machine
Before our pilot started we identified that storing the local copies of source code within changing snapshot states could create problems. At the same time, we found it desirable to put the development tools inside our virtual machines in order to get around remote SharePoint development difficulties and to keep the host build uncluttered. To resolve this conflict, we adopted this approach:

Created a share on the host machine and granted ownership of that directory to a new user account that is used solely for this purpose
Created a new user account with the same name and password in the guest virtual machine
Mapped a drive from the guest to the host's new share, using the internal network's IP address and these new credentials
Launched Visual Studio and downloaded project source code, pointing to the newly mapped drive as the location for the local source
Created a new Code Group for the mapped drive to enable trust for code execution

These steps are covered in more detail in the build guides that will follow in later posts. This is just an outline of the approach to extracting the local code from the snapshot state, which should be adaptable to other development systems with snapshots. For instance, we've found it necessary to download code to a unique location within this directory for each each user, as TFS tracks the network locations that code is checked out to. This is easy to get around, as each user just specifies a unique directory name. If this were insufficient for project requirements, this TFS behaviour could possibly be "fooled" by using NTFS junction points. We've not seen a need for this yet, but we're confident that with this additional option we should be able to store local source code in this manner, and this has been validated by our project experience with Hyper-V to-date.

Summary
These are the high-level design choices that emerged early in the consultancy and research, which have remained largely unchanged to-date. These choices represent one design that has been validated for our needs, which has some shortcomings like any approach. Some of these issues will be covered in the final post in this series.

The information in this post has not covered implementation in any detail, but do not fret. In the next section I will cover the step-by-step build guide for the Hyper-V host laptop.

Building a SharePoint 2007/2010 development environment – Part I: Introduction and Objectives

noreply@blogger.com (Content and Code) — Wed, 04 Nov 2009 13:48:00 +0000

Tristan Watkins, Infrastructure Consultant

Over the last few months I've led the consultancy and system design for a SharePoint 2007/2010 development environment built on Hyper-V in Windows Server 2008 R2. This series of six posts will reveal the key decisions and will consolidate recommendations from a broad range of research and guidance. This first post offers a technology-agnostic introduction to the problem, pros and cons of alternative approaches and what we hoped to achieve with the new approach. The design decisions will be covered in more detail in the second post, followed by a deeper look at detailed build guidance.

Historical development approaches
We have used a variety of development approaches in the past, which have all had limitations and were often an obstacle to productivity. They included:

Shared development farms
Shared farms are appealing at face value because they put development resource hunger on beefier infrastructure. However, there are considerable drawbacks to this approach, including:

The infrastructure burden of these environments is immense
Due to limited physical resources, these servers housed multiple projects in a single farm, leading to hive pollution and a less reliable/predictable development environment
When multiple developers would use the same farm, this would amplify resource contention on already strained servers and it made managing application-level change much more difficult

Local development
Developing SharePoint directly on a Windows Server 2008 laptop introduced a need for frequent laptop rebuilds, as the SharePoint customisations for one project rarely carry over to another and the hive gets polluted. This adds a hefty support burden and frequent down-time for developers.

Virtualised development using Virtual PC
Virtual PC is a friendly technology for SharePoint developers to use, as it does not over-complicate virtualisation, but the management functionality (such as snapshots and snapshot export) is inflexible and performance is poor when compared to a Type-1 hypervisor.

Third-party virtualisation technologies
Most third-party technologies have associated license costs for businesses and many do not support virtualisation of 64-bit guest operating systems. This is problematic because:

Windows Server 2008 R2 is only available in 64-bit
SharePoint 2010 is only available in 64-bit

Additionally, many developers have strong preferences for third-party virtualisation vendors and if this is not controlled/standardised it can introduce incompatibility issues.

Why none of these approaches were satisfactory
In short, these environments did not offer good enough performance or management flexibility and we needed a standardised approach for business continuity. We also needed a technology that would support 64-bit systems.

SharePoint development complications
SharePoint development in teams has always been tricky, as SharePoint virtual machines cannot be easily cloned and renamed. This means that SharePoint environments require:

Scripted installation, which is great for standard builds, but not terribly easy to reconfigure for project-specific requirements, or...
Manual installation, which is time-consuming and not always aligned with a developer’s skill set, or...
Shared environments, which suffer from the drawbacks outlined above, or...
Network isolation of the cloned machine so that that nothing on the network will ever be aware that there are clones

These are some of the biggest issues that need to be confronted when designing a SharePoint development environment. This list is by no means exhaustive; in the final post in this series I'll be reviewing some of the issues that we've encountered during a pilot and subsequent roll-out to all of our developers, consultants and architects.

Distilled objectives
After a thorough review of development approaches, difficulties and available technologies, a Windows Server 2008 R2 (RC) with Hyper-V pilot was announced, soliciting senior developer/architect participation, stating the following aims:

Improve performance over Virtual PC by adoption of a new core virtualisation technology
All available performance guidance for hardware, Windows, IIS, SQL and SharePoint would be considered for inclusion in the build
Hyper-V R2 was adopted early for the pilot (at Release Candidate) to take advantage of further performance gains in the new version
Project cost effectiveness would be improved by:
- Removing development tools and SharePoint from the host machine, reducing the amount of reconfiguration on laptop rebuild and reducing rebuild requests
- Providing a suite of pre-configured virtual environments, reducing setup time at project commencement
- Making project-configured environments reusable within teams throughout the life of the project with Hyper-V’s snapshot technology
- Eliminating the infrastructure burden of shared development farms and re-provisioning those resources to better purpose
Improve the testing process by introducing snapshots

In conjunction with the pilot we upgraded physical disk speed and capacity from 160GB 5400 RPM drives to 300GB 7200 RPM, as Hyper-V snapshots chew up lots of space, and the added spindle speed is critical to improving performance.

Defining these objectives, limiting the duration of the pilot and testing the new approach were critical to gaining management support for the project. In advance of pilot deployment I trained all pilot participants on Hyper-V and introduced our approach to networking and source code storage, which would not be obvious otherwise.

In the next section I will cover the design decisions that were reached through consultancy with the business, pilot findings and research.

[Also posted on Tristan's personal blog]

I'm in love...with the new OpenXML SDK!

noreply@blogger.com (Content and Code) — Thu, 29 Oct 2009 11:51:00 +0000

Tobias Lekman, Technical Architect

I have spent the last couple of weeks creating a Word report based on SharePoint data that spans over a hundred pages using tables and custom graphs.

To do this, I used OpenXML. The file format (.docx) is basically a ZIP file containing a bunch of XML files. I did this using LINQ, XSLT and System.IO.Packaging to compress and order the XML sections into a .docx file. The OpenXML SDK version 1 allows you to manage the packaging side and relationships of the “parts” (XML files).

The new SDK allows you to manage the entire document directly using LINQ which would have bypassed my 60+ XSLT library that I had to create for the original conversion to work. The structure of the program would have been the same but could have saved me a lot of code lines!

When creating OpenXML documents you often end up with corruption. The file is missing a relationship between a diagram and the underlying data, an ID of a part is repeated and so forth. Opening the file in Word will give you a nasty error without any detailed information and you are left to open the document and manually figure out where the corruption could have occurred. The SDK now contains a class called OpenXmlValidator which will validate the entire package and give you a detailed error report. Again, this could have saved me hours and hours of debug time.

To make things even better, the SDK contains some fantastic tools. My favourite is the Document Reflector which opens an OpenXML document from a template and will give you the full code for how to generate that document using the SDK. Now, I don’t even want to begin to estimate how much time THAT would have saved me.

As a final nail in my coffin, SharePoint 2010 introduces REST for Excel Services. My report contained 50+ graph reports which had to be created using 6 different XML parts. This can now be done by inserting the graph generated directly by Excel Services using SharePoint 2010.

The final version of the new SDK will be released together with Office 2010.

Data Sources and Features in Visio Services

noreply@blogger.com (Content and Code) — Wed, 28 Oct 2009 14:19:00 +0000

Zane Freame, Product Manager

Following on from my earlier post, Visio Services in SharePoint 2010, I wanted to delve a little deeper into how this new Shared Service works. Microsoft have been working hard to create a great framework for creating your own data driven visualisation in SharePoint. Here's a high level look at how they're helping you do it.

Diagrams can be published directly to SharePoint document libraries and either opened in the browser directly from the file, or published in full fidelity using a Visio Web Access web part. Both methods use Silverlight to render the diagram, and for those who don't have Silverlight, the diagrams can be published as a PNG image minus a few interactive features.

According to the guys from Microsoft, the published diagrams are browser agnostic and will work in Internet Explorer, Firefox and Safari to name a few. Data can be refreshed manually or automatically with a variety of options in the web part to customise what users can see and do.

Visio Services for SharePoint 2010 supports a variety of data sources including:

SQL (ODC)
Excel workbooks (via Excel Services)
SharePoint Lists
OLEDB
ODBC
Custom (not sure what this is yet, but I'm sure I'll find out soon!)

The features don't end there either. I haven't had a chance to build my own (yet!) but here are just a few of the features and tools that you can expect to see in the final product.

Ability to pull data from multiple data sources on the same diagram
Zoom in/out functionality
Automatic refresh of data
JavaScript Mash-up API
Support for web part connections

One disclaimer... although you can still create diagrams in Microsoft Visio 2007, in order to publish them to SharePoint 2010, you will need Microsoft Visio 2010 installed on the client computer. Of course, users just viewing the diagrams in SharePoint will not require Visio installed.

For more information about Visio 2010 visit the official Visio Blog on MSDN - Visio Insights.

SharePoint 2010: Architecture Guidance - things everyone should know!

noreply@blogger.com (Content and Code) — Tue, 27 Oct 2009 12:03:00 +0000

Martin Hatch, Solutions Architect

Well, the final day of the conference came and with it some of the most useful sessions (from my perspective). One of which was the "Architecture Guidance for SharePoint 2010". This hopefully distils some of that information. It's not a be all and end all, but hopefully points you in the right direction so that you can focus your research a little better!

UI Design

Entire interface in SharePoint 2010 to be W3C XHTML compliant
SharePoint 2010 "more accessible mode" to be WCAG 2.0 AA compliant
New ribbon interface replaces toolbars and menus (and considerations for old "CustomAction" commands which may no longer work!)

Wiki content allows web parts to be dropped in (removing over-reliance on web part zones)

Lists

There are a whole load of new List capabilities (in addition to the "External List" that BCS brings to the plate!).

Lookup to Multiple

This means that when you create a new lookup column, you can now pull down additional fields from the lookup list item and use them for filtering.

CAML support for Joins!

You can now perform "JOIN" operations in your CAML queries for linking lists together.

Enforced List Relationships

You can now enforce specific relationships for lookup columns with two options:

Restrict Delete - cannot delete parent if child items exist.
Cascade Delete - If you delete the parent, all child items are automatically deleted (recycle bin aware with "restore" options!)

Store-level enforcement

This is code level "required fields", so now you can enforce the requirements even through code !

Unique Fields

Specify a unique field, so that no two values can match (e.g. Email addresses in contacts list)

Compound Indices

If you want to query by 2 fields, you can now index both at once as a compound index.

<In> clause for reverse lookups

This allows a CAML query to do a reverse lookup to get all child items that are associated with the parent!

Formula based validation

e.g. Don't allow Field2 to be lower than Field1.

Workflows

Out of the box SharePoint 2010 workflows can now be extended in SharePoint Designer 2010.
SharePoint Designer 2010 can be used to create "re-usable" workflows
Site Workflows - to manage processes across an entire site.
You can now import a SharePoint Designer 2010 workflow into Visual Studio 2010!
Import/Export workflow using Visio 2010 for visual workflow modelling.

Content & Document Management

"Document Sets" allow you to treat a group of documents as a single item (with 1 version history, group executed workflow and policy, and a "download as zip" option).
Managed Metadata Service allows cross-farm Content Type management. This is a killer-app, bringing true enterprise content management to SharePoint 2010.
Enterprise Wiki's allow more rapid "in edit" content, as well as Web Parts deployed directly into the rich text editor (no more web part zones?).
Spelling check and broken link check when you "check-in" WCM pages.

Event Handlers

Three new event handlers added (at last!!)

WebAdded - Fired every time a child site is created in the web.
ListAdded - Fired every time a list is created in the web.
Feature Upgrading - Fired when a feature has it's "upgrade" method called (more on this in a future blog post).

Security

Editing of ASPX pages now required "Designer" permissions (instead of contribute).
XSS (Cross Site Scripting) protection for pages and web parts.
HTML pages will now "force download" by default. This stops people from uploading HTML files with malicious scripts, so if you click on an HTML file in a document library you will get a download dialog instead of the file opening in the browser!
There are still no field level permissions (it was estimated that this would add a 30% overhead to performance! Maybe in a future release)

BI and Connectivity

New Business Connectivity Services (BCS) allows no-code connections of databases and LOB systems to content types and lists with two-way synchronisation of data and full CRUD support.
PowerPivot for Excel allows upwards of 100 million rows into an excel workbook with phenominal performance.

Office Application Support

New web level services for applications (Excel / Visio with JavaScript events!)
SharePoint Workspace to replace "Groove" for offline file support and editing.
Office Web Applications to allow for direct opening and editing of documents from within the browser!

Databases

Still a 100GB "limit" for content databases.
Still cannot have site collections spanning multiple databases.
New support for "Failover" databases, SharePoint 2010 is now SQL mirror aware!
All "Service Applications" have their own SQL database, along with many other new databases (e.g. Feed Activity, Social Data, Usage Logs).
New "read only content databases" open the door for simple content deployment (utilising SQL log shipping or database replication).

Content Deployment

All execution now in Timer Jobs.
Performance (and memory usage) improved.
Export routine now creates database snapshot to improve data integrity!

Sandboxed Solutions

Ability to upload WSPs directly into the content database to execute in minimal permissions using "virtual files" (no impact on the file system!)
Resource throttling, code performance checking and "bad routine" blocking
Provides new best practice for code development and deployment!

I'm sure there are many other things, so please let me know if there's anything else you think should "make the grade" and I'll see if I can add it in :)

[Also posted on Martin's personal blog]

My Site 2010 Profile Pictures Centrally Managed

noreply@blogger.com (Content and Code) — Wed, 21 Oct 2009 06:25:00 +0000

Glyn Clough, Solutions Consultant

Responding to what most surely be one of the most common questions and requests from users of previous SharePoint My Sites, Microsoft have updated the manner in which profile pictures are managed within SharePoint 2010 My Sites (actually the User Profile Database).

The key areas of the new approach are:

Pictures are now stored centrally within the My Site Host site collection in a picture library located http://host/userphotos.
Pictures are resized three ways when uploaded:

32x32 pixels for use within SharePoint 2010
48x48 pixels for use within Active Directory and other client applications (such as Outlook and Office Communicator)
96x96 pixels for use on the My Profile page of a My Site

The Picture Picker can be customised or replaced to support feature such as requesting usage consent or providing default images.

Managing profile pictures was a common headache in SharePoint 2007 and given all the new social features of SharePoint 2010, it's great news that this has been addressed.

[Also posted on Glyn's personal blog]

SharePoint 2010 brings versioning to Web Parts

noreply@blogger.com (Content and Code) — Tue, 20 Oct 2009 23:28:00 +0000

Tobias Lekman, Technical Architect

In the upcoming release of SharePoint Server 2010 we will see versioning of web parts and their properties.

Why is this important?

In the past we had full versioning and approval workflows around the contents of items and pages but any change to web parts and their properties were instantaneous. This caused a lot of confusion and aggravation as web parts, their configurations and content were lost irretrievably when users made changes to them.

All web parts within the page are now fully version controlled with the rest of your page and will not require any additional code within the web part itself.

Visio Services in SharePoint 2010

noreply@blogger.com (Content and Code) — Tue, 20 Oct 2009 23:13:00 +0000

Zane Freame, Product Manager

As many of my colleagues will testify, I've had a long love affair with Visio. It started when I was at University in Melbourne over 12 years ago. I stumbled across a free student version in a local computer store and from that point forward I've been creating all my flowcharts, processes, layouts and many other diagrams in Visio. In fact, in every job I've had, I've found a new application for the product. Content and Code has been no exception. On my first day I discovered that the team were already using Visio to create quick and easy wireframes...so I then went about creating new templates and stencils/shapes to meet our wireframing needs for website development.

Visio is a brilliantly simple visual tool with advanced options for integrating with external data, but until now the missing link has been publishing to SharePoint. Several years ago Content and Code developed a Visio Viewer web part for SharePoint 2003 which enabled users to view Visio diagrams in a SharePoint page provided they had the Visio Viewer add-on installed on the machine. Although that was a great option for many companies, it was no holy grail either.

With Excel Services bringing the power of publishing spread sheets and charts to SharePoint, it was only a matter of time before Microsoft decided to do the same with Visio. Well, there's no more waiting... Visio Services is here! What's more they've packed it with great data interaction features which will blow your socks off! Trust, me if you're a visual person like me, Visio Services for SharePoint 2010 is your bag of fun and will take your Visio diagrams to the next level.

From the slides, Microsoft are proud to point out that with Visio Services for SharePoint 2010 you can:

Share Diagrams in SharePoint
Build Data Driven Visualisations
Integrate with SharePoint Applications

But what does this mean?

I think the best way to explain this is with some real-life examples which most of you would have come across in the past.

A restaurant seating plan indicating which tables are availble before you make a reservation.
Flight seating plan to check which seats are available when you check-in online.
Elections results on a map (e.g. votes per state/county) - picture the election TV broadcast with data presented in blue or red areas on a map with bar graphs and numbers indicating which political party is winning the election race.

These are just a few examples that are out there which you could easily reproduce in Visio Services with little or no development.

It's clear to see the benefits of visualising data in this way, and now with Visio Services we can combine data in SharePoint and other data sources with our Visio diagrams and publish them to SharePoint.

The presenters here at the SharePoint Conference had plenty of other great examples to demonstrate during the sessions I attended (factory floor plans, manufacturing cycles, hotel vacancies, insurance loss per state, ski lift status etc.) proving that the opportunity with Visio Services is only limited by your visual imagination.

Silverlight and SharePoint 2010

noreply@blogger.com (Content and Code) — Tue, 20 Oct 2009 20:09:00 +0000

Jason Visser, Technical Architect

Development of Silverlight and SharePoint has always been very painful however with SharePoint 2010 Silverlight integration has been simplified greatly.

Why use Silverlight instead of other technologies:

It is easier to build for .NET developers as it uses a common language.
Less browser compatibility issues - either the Silverlight app renders or it does not.
More powerful animations, algorithms and a rich development API.
Very powerful client integration, e.g. out of browser support, offline mode, multi touch and access to the local file system.

Silverlight should not be used everywhere and samples of that would be:

When you need to capture or render rich text or html.
When there is large amounts of content that will be paginated and referenced by URL's.

There are 4 main ways to develop Silverlight applications with SharePoint 2010 (these can all be combined if required):

Read and Write to SharePoint using Web Service's.
Read and Write to SharePoint using the new client object model.
Read and Write to SharePoint using REST
Load data from SharePoint by rendering data on the page.

My current preferences would be using REST and the new client object model to read and write to SharePoint.

Silverlight and Web Service integration

SharePoint has always shipped with built-in web services. These include reading and writing to different areas of SharePoint 2007. SharePoint 2010 is no exception and it too ships with various web services.

A quick sample of the type of web services available would be getting lists, list items, sites, navigation, security and search. The exposed web services all make use of the in-built SharePoint security trimming.

If you do not find a web service that works for you, you can build your own web service and call that from your Silverlight application. When developing Silverlight applications using these web services, you will be using asynchronous execution and will be retrieving your result in xml form. These results can then be interpreted and manipulated using various different .net technologies. My favourite would be XLINQ which was the preferred method for reading and writing to SharePoint 2007.

Silverlight and the Client Object Model

New for SharePoint 2010 is the Client Object Model. The client object model for Silverlight is basically a subset of some of the objects that were previously not available to be used by the Silverlight API.

Object's available from the Silverlight client object model include:

Site

ContentType
List

Form
View
Field
ListItem

Folder

File

WebPart

Navigation

NavigationNode

UserCustomAction
RoleDefinition
RoleAssignement
WorkflowAssosiation
WorkflowTemplate

The client object model uses asynchronous call's to read and write to the SharePoint site in batches. There are 2 assemblies that can now be found in the SharePointRoot\14\template\layouts\clientbin folder that include the object described above. The client object model uses a context similar to the SPContext object, this is called the ClientContext. An example of how to code using this model is:

ClientContext context = ClientContext.Current();
Web = context.web;
context.load(web);
context.load(web.lists);
context.ExecuteQueryasync(this.ExecuteMethod, null);

Silverlight and REST

You can read and write to SharePoint using REST which creates strongly typed object using the REST services. It allows you to code against SharePoint using OOP and ATOM-based means. This and the client object model are the preferred ways to communicate with SharePoint.

Silverlight and rendering data on the page as initialisation parameters

This is the oldest way of providing your Silverlight application with SharePoint data. Initiation Parameters can only read data from SharePoint using the Silverlight 'init params' method. Using this method you need to print out the data that you need to load into the Silverlight application. If there is a lot of data it will all need to be printed to the page / Silverlight object. This can be useful for initial Silverlight setups like theme settings and view settings, rather than loading up a default skin and waiting for the response from SharePoint to load up your initialisation parameters.

Sandboxed Solutions

noreply@blogger.com (Content and Code) — Tue, 20 Oct 2009 20:07:00 +0000

Tobias Lekman, Technical Architect

One of the largest paradigm shifts for developers in SharePoint 2010 is the introduction of “sandboxed solutions”.

“A sandboxed solution is a new concept in Microsoft SharePoint Foundation that allows site collection users to upload their own custom code solutions. A sandboxed solution uses a subset of the Microsoft.SharePoint namespace. These objects are marked in the object model to show their availability in a sandboxed solution”

Source: http://msdn.microsoft.com/en-us/library/ee539083(office.14).aspx

Simply put – you should always use a sandboxed solution until you can prove that a sandboxed solution is not sufficient to get the job done.

A sandboxed solution works in the same way as a normal solution (farm level solution) but runs in minimal privilege mode. You cannot place files within the old “hive” but rather work with virtual files. It also runs within a separate process outside of the SharePoint application and communicates with SharePoint 2010 over a proxy.

On the upside: your code will not conflict with other custom solutions. On a shared environment this is a huge benefit. The system is also protected from bad code as detection of abnormal CPU, thread or memory usage will cause the component to be booted out of the system.

On the downside: It means more work for developers.

When you need to access external systems, you will now write a “full-trust proxy” that is installed in GAC and made available to all sandbox solutions. This class will have a contract describing exactly what additional access the code can gain. For example, you can read information from a CRM database, but not add data nor access any other external database.

This means less risk of installing third party and bespoke components, less need for code reviews and easy monitoring of software components.

A common breakdown of your solution might be:

List definitions, fields and content types (sandbox)
Code-only Web parts (sandbox)
Full-trust proxy solution with contract for access to read CRM (GAC)
Data layer for CRM (GAC)

For a complete architectural view of sandboxed solutions, see http://msdn.microsoft.com/en-us/library/ee539417(office.14).aspx.

Top 10 (+1) new things in SharePoint Designer 2010

noreply@blogger.com (Content and Code) — Tue, 20 Oct 2009 13:59:00 +0000

Glyn Clough, Solutions Consultant

I attended a great session on day one of the SharePoint Conference by a guy called Asif Rehmani, MVP MCT. Asif is the co-founder of SharePoint-Videos.com and a mixture of trainer and solution architect for SharePoint and perhaps it was this training background that helped him deliver such a fantastic session. I strongly recommend checking out SharePoint-Videos.com as there is already SPD 2010 content up there!

The introduction to SharePoint Designer 2010 was a demo of the top 10 cool new things that you can do in the product. Here they are:

Restrict SPD as needed
It's possible to set permissions at the web application or the site collection level as to who can use (and what they can do) SPD 2010.
New user experience with summary pages, ribbon and quick launch navigation
SPD 2010 has had a re-design along with the rest of office. The key element for me is that it is now a highly integrated experience with SharePiont. For example you can browse the content types and site columns available in a site direction from SPD.
Create SharePoint content
As you'd expect you can create new pages, sites, lists etc all from within SPD.
Configure site security
Previously it was only possible to configure the permissions of a site through the browser, but now SPD also provides this functionality.
Create new content types and attach to lists directly
Create site assets to store files needed for the site
There's a new type of library called 'site assets' in SharePoint 2010 and this can be populated via SPD. Assets such as CSS, XML or JavaScript files can be stored here.
XSLT List View Web Parts... Everywhere!
A new web part for 2010 is the XSLT List View Web Part which is a mix of the old XSLT Data View Web Part and the List View Web Part from 2007. This provides fantastic functionality such as conditional formatting etc in a really well integrated web part - that can now also be configured via the browser.
Connect to data sources outside of SharePoint
Databases / XML files / Server-side scripts / Web Services are all external data sources that can be easily dropped into a page and presented to the user via SPD.
Create Business Connectivity Services external content types
The BCS is the evolution of the Business Data Catalogue and has been fully integrated into SPD. It's possible to quickly, with no code, link to an external data source (e.g. SQL) and present this information as an 'External Content Type' in SharePoint 2010. This can then be viewed and edited in SharePoint via a new list type 'External List'.
Create powerful reusable workflow
There are now three types of workflow available in SPD: List Workflows, Reusable Workflows, Site Workflows. The impressive thing is that SPD workflows now no longer need to be tied to a specific list in a specific site! They can also be saved out as a WSP or exported to Visio.

These are all fantastic features - and the implicit one in all of them is the great usability of SharePoint Designer. As with other areas of the SharePoint experience, there have been huge efforts at increasing productivity. It's obvious to see that its matured as a product, to such a level that (pending further investigation!) I'd be comfortable recommending using SPD for day-to-day activities of site administration - on the production environment!

[Also posted on Glyn's personal blog]

Standards compliant SharePoint - too good to be true?

noreply@blogger.com (Content and Code) — Tue, 20 Oct 2009 03:49:00 +0000

Martin Hatch, Solutions Architect

Well it seems not. Microsoft have really pulled out all of the stops this time around, with standards compliant, cross browser compatibility and even accessibility hitting the conference headlines.

The first item of note is that SharePoint 2010 aims to be W3C XHTML compliant. This is a massive leap towards making SharePoint 2010 a truly robust platform, worthy of both internet facing sites and global internal systems.

I saw a demo of an embedded video in the Monday afternoon Web Content Management session where a video was streamed from a document library (another new SharePoint 2010 feature) and it was loaded in FireFox with no plugins or downloads, full screen streaming and buffering.

But the biggest impact is bound to be around the statement that one of the major release goals for the SharePoint 2010 release is that the entire interface of SharePoint 2010 meets WCAG 2.0 AA compliance! This includes editing and authoring of content! This is absolutely massive, and represents a huge push from Microsoft to really respond to the community feedback that accessibility is a subject to be taken seriously.

Of course, there is bound to be community discussions around the ideology that "compliant" is not the same as "accessible" (which we at Content and Code know all too well, having developed several fully accessible SharePoint 2007 solutions for the Royal National Institute of Blind People ) but this is still a massive step forward and certainly shows a continuing respect of standards from Microsoft and good news for the future of compliant software.

[Also posted on Martin's personal blog]

Key notes from the Keynotes

noreply@blogger.com (Content and Code) — Tue, 20 Oct 2009 03:37:00 +0000

Martin Hatch, Solutions Architect

The keynote speeches contained a huge amount of content, but there were some standout points for me that were worth condensing into a post...

One of the main aesthetic changes is that Windows SharePoint Services (WSS) is now called SharePoint Foundation 2010. Hopefully more details about capabilities will surface during the conference.
There was much talk about SharePoint Online. They apparently have over 100,000,000 users and the platform is updated quarterly with new functionality, so we can hope to see new SharePoint 2010 functionality in that creeping through once the RTM version of SharePoint is released.
The list item storage limits has gone WAY up ... 1,000,000 items per folder/list and over 10,000,000 documents per library.
Another favourite was how Excel Services in SharePoint 2010 allows you to expose excel data as REST feeds (such as charts, tables, images, pivot tables). This allows you to subscribe to an image URL which is actually being dynamically generated from the Excel 2010 spread sheet. The main focus here was that you can embed this image anywhere that a normal HTML or Office Client image can be placed, but if the Excel 2010 spread sheet data is modified then the image is automatically updated to reflect those changes!
Other highlights included forcing spelling checkers and broken link checkers on check-in of a page. In fact there was a lot of mention around web sites full stop, particularly 2 new products for websites: SharePoint Server 2010 for Internet Sites and FAST Search Server for Internet Business.
The new Wiki Editing features were demonstrated with auto-complete URLs for lists, views and folders in libraries.
For those power users there are also over 500 new PowerShell commands for SharePoint 2010 which will be shipped with the Beta version in November. These can even be run on a Windows 7 machine and executed remotely!

But by far the most impressive part for me was the presentation on SQL Server PowerPivot for SharePoint 2010 and SQL Server PowerPivot for Excel 2010. This is the product formerly known as "Gemini" and allows you to pull data in from SQL databases to allow up to 100,000,000 rows of data in Excel 2010! You can then filter, sort and produce charts which refresh near instantaneously! There was a demo of this in the session causing a round of applause from pretty much everyone.

A very interesting session in the end, with glimpses of promise from all over the platform. I can't wait to get my hands on it!

[Also posted on Martin's personal blog]

The strategic evolution of SharePoint

noreply@blogger.com (Content and Code) — Tue, 20 Oct 2009 03:02:00 +0000

Glyn Clough, Solutions Consultant

During his keynote session at the SharePoint Conference 2009 Jeff Teper, Corporate Vice President for SharePoint, showed an interesting PowerPoint slide from 1999 when Microsoft were first strategising about a new product that was to become SharePoint. The amazing thing is that 10 years later the key elements of the strategy set down then are now just as applicable now:

Provide a great integrated solution
Out of the box work spaces
Compelling Office integration
Easy and flexible deployment

The good news for all of us SharePoint professionals, and more importantly the millions of SharePoint end-users, is that this shows a consistent long-term approach to the product that has definitely helped shape where we find SharePoint 2010 today and hopefully where it will go in the future!

[Also posted on Glyn's personal blog]