Archive for the 'Uncategorized' Category

Running PowerShell to change MSBuild scripts with NuGet

Saturday, February 4th, 2012

One of the things I’ve missed from creating a number of NuGet packages, is not being able to add msbuild tasks to the .csproj file.  From running web.config transforms, versioning assemblies, to running unit tests, code analysis, or deployments – msbuild can add quite a lot of cherries to your builds.  In a web agency, I work with a lot of projects for fairly short periods of time, and even copying and pasting all those useful scripts means they often get missed / take too much time, or just plain break because I’ve forgotten something.

Re-using code still can take hours out of my project which I’d much rather be using to create something new.  Combine that with a larger team of people doing the same thing, and we need a better solution.

NuGet makes it very easy to install binaries or content/src files, and the nuspec syntax is beautifully simplistic.  In it’s absolute barest form, it looks a little like this:

<package xmlns="http://schemas.microsoft.com/packaging/2010/07/nuspec.xsd">
  <metadata>
    <id>VersionAssembly.Mvc</id>
    <version>1.6</version>
    <authors>Kevin Blake</authors>
    <description>Version Assembly</description>
  </metadata>
  <files>
    <file src="VersionController.cs.pp" target="content\Controllers" />
  </files>
</package>

That doesn’t look like very much, but when we create a NuGet package (nuget.exe pack .), we’re going to get a package that will install a new controller, and the .pp file extensions means that file will be processed to include the correct namespace for our project.  In this case, the controller just outputs the assembly version number – but that’s not important.

We can also add dll’s to this, with target=”lib”, for all those libraries you’ve got hanging around on your server.  More excitingly, you can run PowerShell scripts along with your NuGet install, by using target=”tools\install.pl1″.  PowerShell is a very powerful scripting language from Microsoft (yep, another one), which among many other things, can give your NuGet packages all the extra power to do what you like…

Let’s start off with a really basic script that’ll run while installing your package by adding to your <files> node within your nuspec.

<file src="install.ps1" target="tools\install.ps1" />

Note that you can only have an install script if you’ve already got a content or lib file in there as well.  So go and add one (even a package_readme.txt) if you haven’t already.  You can also use init.ps1 instead, but that’s going to run every time your project loads.  For our sake, the controller we already have does the job just fine.

Now you’re going to need a PowerShell script, so let’s start out small with Hello World.

Write-Host "Hello World"

You can probably guess that this won’t do very much, but you should be able to see that output when you install you NuGet package.  You also have at your command, your simplest form of echo/print debugging tool, for PowerShell beginners like me.

With that working, you can move onto something a little more useful… The following script will add a Version task to your .csproj file, which updates the Properties\AssemblyInfo.cs with the version number that (in our case) comes from CruiseControl.Net.  Most of the work for this task comes with the MSBuild Community Tasks, so you will need that installed if you’re going to try this out word-for-word.

param($installPath, $toolsPath, $package, $project)
 
$buildProject = Get-MSBuildProject
$import = $buildProject.Xml.AddImport("`$(MSBuildExtensionsPath)\MSBuildCommunityTasks\MSBuild.Community.Tasks.Targets")
$target = $buildProject.Xml.AddTarget("Version")
 
$target.BeforeTargets = "BeforeBuild"
$target.Condition = "'`$(Configuration)' == 'Release'"
$task = $target.AddTask("AssemblyInfo")
$task.Condition = "'`$(CCNetLabel)' != ''"
$task.SetParameter("CodeLanguage", "CS")
$task.SetParameter("OutputFile", "properties\AssemblyInfo.cs")
$task.SetParameter("AssemblyVersion", "`$(CCNetLabel)")
$task.SetParameter("AssemblyFileVersion", "`$(CCNetLabel)")
 
$buildProject.Save()
$project.Save()

That Get-MSBuildProject statement is actually supplied by another NuGet package, so we’ll have to change our NuSpec file to include that as a dependency by adding to the metadata block.

<dependencies>
      <dependency id="NuGetPowerTools" version="0.26" />
    </dependencies>

It’s also a good idea to add an uninstall.ps1 script that will undo all of those changes if someone later decides not to use your package.  I’m not going to cover that here because it’s not something I’ve completed, and these packages will not be publicly posted (by all means, share your examples in the comments, if you have).

Your complete NuSpec file should now look a little bit like this

<package xmlns="http://schemas.microsoft.com/packaging/2010/07/nuspec.xsd">
  <metadata>
    <id>VersionAssembly.Mvc</id>
    <version>1.6</version>
    <authors>Kevin Blake</authors>
    <description>Version assemblies</description>
    <dependencies>
      <dependency id="NuGetPowerTools" version="0.26" />
    </dependencies>
  </metadata>
  <files>
    <file src="VersionController.cs.pp" target="content\Controllers" />
    <file src="tools\install.ps1" target="tools\install.ps1" />
</files>
</package>

It’s a very simple example, I know – but these basic steps give almost limitless possibilities to your NuGet packages… Personally, I will be creating a lot more of these to include build scripts we would otherwise have had to include in a rather large bloated project template.  NuGet+PowerShell gives the flexibility to choose just what we need from the dessert cart, and that can only be a good thing.

Using Umbraco with Web Application projects, CI, across teams

Saturday, January 28th, 2012

Can I use just a Web Site project?

Yes – of course.

I don’t really want to get into the debate about Web Site projects, vs. Web Application projects.  But I do want to start off by saying I dislike Web Site projects, and we’d never think about using them anywhere else.

One thing they offer though, is all of the files you drop into the directory become part of your Visual Studio solution.  This is really useful for applications such as Umbraco where developers are uploading media items, adding scripts, or creating templates via the Umbraco back-end.  It’s a pain to have to switch back to Visual Studio and add those generated files to your project, just to ensure they get published with the rest of your site.

We use a CI server, we want repeatable deployments, we’re working in teams, we want everything that’s anything to be in source control, we want the moon on a bloody stick – and Web Site projects just don’t cut it.  But I don’t really want to even mess around with everyone having to only work in Visual Studio either, so PostBuild events on separate projects that copy in razor scripts / templates etc is also – no good.  The beautiful thing with Umbraco is it’s completely flexible, there’s no right or wrong way to use it.  So here’s how we’re (currently) managing Umbraco (v4) builds.

Getting started with your web application

Firstly, create a new Empty Web application project in Visual Studio.

Download and copy all of the files from the latest Umbraco release into your project.

Don't forget the show all files button, to see everything in your directory

Now in Visual Studio, you want to add all the directories / files to your solution that you aren’t going to let people manage through Umbraco as well.  For our purposes, that’s config, data, App_Data, App_Browsers, data, umbraco, umbraco_client, default.aspx, and web.config.  If it’s still useful to have some of those other files/directories in your project later, don’t worry – you can add them (and have Umbraco/msbuild manage the rest).

You can add the bin folder as well, if you want – but we don’t like all binary files checked into sourcecontrol, so this is one folder we usually exclude by default.

If you want the same, copy all of the files out of the bin folder to a directory outside of your csproj (call it Resources, Libraries, whatever you like).  Then in Visual Studio, Right-click on your references folder, Add reference, and include all those dlls.  You’ll need that Resources / Libraries folder included in sourcecontrol as well.

You now have a solution that will build and work locally.  But you still don’t have the ease of including new Umbraco-managed items in your builds (try and right-click to publish your project to the file system, and notice how you’re still missing all of your media items and masterpages).  That’s no good for our Continuous Integration server, and a Web Site project would include all those items by default (you’d need to create one of those Web Deployment project as well though).

Modify MsBuild scripts… Include everything the wrong way

Msbuild to the rescue.  Open your .csproj file in your favourite editor (unload the project in Visual Studio if you want to use that).

You’ll see a whole bunch of xml nodes like <Content Include=”default.aspx”>.  These tell msbuild to include these files as Content items, and copy them to the Output directory on every build.

You can use wildcards, so you could just add a new item such as <Content Include=”masterpages\**\*.master” /> and you’d think that would include everything.  And you’d be right – reload your solution in Visual Studio,  and all of those masterpages will be included.  However – exclude one file, re-add it, and save your project.  If you inspect your .csproj file again – look how helpful Visual Studio has been… It’s replaced your wildcard map with a list of individual files again.  Bad Visual Studio.

Modify MsBuild scripts… Include everything the right way

Since that’s clearly going to go wrong somewhere down the line, one of your developers is going to do exactly that and unknowingly bring your entire build solution crumbling to its knees – this clearly won’t do.  There is always another way.

We can modify the BeforeBuild event (that’s currently commented out near the bottom of your csproj file, so uncomment it and replace it with this.

<Target Name="BeforeBuild">
<CreateItem Include="macroScripts\**\*.cshtml">
<Output ItemName="Content" TaskParameter="Include" />
</CreateItem>
<CreateItem Include="xslt\**\*.xslt">
<Output ItemName="Content" TaskParameter="Include" />
</CreateItem>
<CreateItem Include="masterpages\**\*.master">
<Output ItemName="Content" TaskParameter="Include" />
</CreateItem>
<CreateItem Include="media\**\*.*">
<Output ItemName="Content" TaskParameter="Include" />
</CreateItem>
<CreateItem Include="usercontrols\**\*.*">
<Output ItemName="Content" TaskParameter="Include" />
</CreateItem>
<CreateItem Include="css\**\*.*">
<Output ItemName="Content" TaskParameter="Include" />
</CreateItem>
</Target>

What we’re doing is adding to the Content array that msbuild’s defined with all those Content nodes.  We’re including all files from those directories in the publish, and we’re making the Web Application project work the way that Umbraco, and we want.  You can include as many of the folders as you’re comfortable with, and you can continue to work in Visual Studio for as much or as little of the sln as you want.

Avoid doing this next time

Having putting in this small amount of work up-front, you can now export your entire project as a project template, and use it to start off all your projects in the future.  You probably have more of a white label sln you can add to this and make your subsequent builds even easier.  I won’t get into the details of that in this post (but it’s essentially just File-Export Template within Visual Studio).

Like I said, Umbraco is completely flexible in the way you approach your builds, this is one solution that works for us – and there is always another way.  I’d be keen to hear your feedback / improvements / alternatives – we haven’t etched it in stone by any means.

Similar posts that have inspired, and Another Way:

(Send me more, and I’ll add to this list)

git out of memory errors – git repack

Saturday, January 7th, 2012

Occasionally when using git (particularly with large repositories) you might come across an error that looks a little bit like this:

remote: Counting objects: 506, done.
remote: fatal: Out of memory, malloc failed (tried to allocate 342222798 bytes)
remote: aborting due to possible repository corruption on the remote side.
fatal: protocol error: bad pack header

It means the remote server has run out of memory while packing objects on the server ready to send down the wire, and has aborted the pull request mid-way.  If you know you’ve got some particularly massive files in your repository that you didn’t really need there, you could rewrite history and get rid of them, but chances are you just need to do a little maintenance on your remote repository.

Connect to your remote server, and find the repo in question, then just run the following:

git repack -a -f -d

Now you should be able to pull your repo again, with everything already packed on the server.

Obviously, you can find more about those options with:

man git-repack

//TODO: Feature Release for v2012

Sunday, January 1st, 2012

I did one of these back in 2009 and after 2 years, I can successfully say I’ve accomplished… some of them.  It’s the sort of list I’d make an annual review if they didn’t have to follow fluffy company ideals and goals.  Not a new years resolution as such – but technologies and platforms I want to be looking at in the near future.

  • Umbraco
    • v5, in general
    • Custom v5 Hive Providers
    • Courier extensions
    • Contour extensions
  • ASP.NET MVC 4
    • Mobile Views
  • Entity Framework Code First
  • .NET 4.5
    • New async features
    • WebSockets
    • Contract-First WCF
  • T4 Templating
  • Resharper 6.1 Shared Settings
  • NuGet – more of it.  NuGet the f**k out of everything.
  • KnockoutJS
  • Gallery2/3
  • Amazon MWS.
  • Zencart, Magento, or whatever is passing as a popular choice of shopping cart nowadays

 

Farewell 2011, Long Live 2012

Sunday, January 1st, 2012

I’ve seen tons of 2011 retrospective blog posts over the past month. Not having one is making me feel inadequate, so gathered from various stats on my machine, and social networks – last year, I:

My most-listened to artists from 2011

  • Read 18 novels.
  • Listened to 17,650 tracks.
  • Drank 102 beers (88 unique) since August… That I checked into.
  • Wrote 3 blog posts (must try harder next year – already a third of the way there).
  • Visited 3 new countries – grand total distance between the capitals and London of approx. 26,658km.
  • Took approx. 7,597 photos.
  • Uploaded 1,249 photos to Blakepics.
  • Recorded 667 TV shows with MythTV, totalling 20 days, 21 hours of viewing.
  • Pinned 94 pictures on Pinterest.
  • Tweeted 553 times.
  • Played 28 xbox games
  • Shared far too much information with social networks, and online services.

What have I missed?