Running PowerShell to change MSBuild scripts with NuGet

Saturday, February 4th, 2012

One of the things I’ve missed from creating a number of NuGet packages, is not being able to add msbuild tasks to the .csproj file.  From running web.config transforms, versioning assemblies, to running unit tests, code analysis, or deployments – msbuild can add quite a lot of cherries to your builds.  In a web agency, I work with a lot of projects for fairly short periods of time, and even copying and pasting all those useful scripts means they often get missed / take too much time, or just plain break because I’ve forgotten something.

Re-using code still can take hours out of my project which I’d much rather be using to create something new.  Combine that with a larger team of people doing the same thing, and we need a better solution.

NuGet makes it very easy to install binaries or content/src files, and the nuspec syntax is beautifully simplistic.  In it’s absolute barest form, it looks a little like this:

<package xmlns="http://schemas.microsoft.com/packaging/2010/07/nuspec.xsd">
  <metadata>
    <id>VersionAssembly.Mvc</id>
    <version>1.6</version>
    <authors>Kevin Blake</authors>
    <description>Version Assembly</description>
  </metadata>
  <files>
    <file src="VersionController.cs.pp" target="content\Controllers" />
  </files>
</package>

That doesn’t look like very much, but when we create a NuGet package (nuget.exe pack .), we’re going to get a package that will install a new controller, and the .pp file extensions means that file will be processed to include the correct namespace for our project.  In this case, the controller just outputs the assembly version number – but that’s not important.

We can also add dll’s to this, with target=”lib”, for all those libraries you’ve got hanging around on your server.  More excitingly, you can run PowerShell scripts along with your NuGet install, by using target=”tools\install.pl1″.  PowerShell is a very powerful scripting language from Microsoft (yep, another one), which among many other things, can give your NuGet packages all the extra power to do what you like…

Let’s start off with a really basic script that’ll run while installing your package by adding to your <files> node within your nuspec.

<file src="install.ps1" target="tools\install.ps1" />

Note that you can only have an install script if you’ve already got a content or lib file in there as well.  So go and add one (even a package_readme.txt) if you haven’t already.  You can also use init.ps1 instead, but that’s going to run every time your project loads.  For our sake, the controller we already have does the job just fine.

Now you’re going to need a PowerShell script, so let’s start out small with Hello World.

Write-Host "Hello World"

You can probably guess that this won’t do very much, but you should be able to see that output when you install you NuGet package.  You also have at your command, your simplest form of echo/print debugging tool, for PowerShell beginners like me.

With that working, you can move onto something a little more useful… The following script will add a Version task to your .csproj file, which updates the Properties\AssemblyInfo.cs with the version number that (in our case) comes from CruiseControl.Net.  Most of the work for this task comes with the MSBuild Community Tasks, so you will need that installed if you’re going to try this out word-for-word.

param($installPath, $toolsPath, $package, $project)
 
$buildProject = Get-MSBuildProject
$import = $buildProject.Xml.AddImport("`$(MSBuildExtensionsPath)\MSBuildCommunityTasks\MSBuild.Community.Tasks.Targets")
$target = $buildProject.Xml.AddTarget("Version")
 
$target.BeforeTargets = "BeforeBuild"
$target.Condition = "'`$(Configuration)' == 'Release'"
$task = $target.AddTask("AssemblyInfo")
$task.Condition = "'`$(CCNetLabel)' != ''"
$task.SetParameter("CodeLanguage", "CS")
$task.SetParameter("OutputFile", "properties\AssemblyInfo.cs")
$task.SetParameter("AssemblyVersion", "`$(CCNetLabel)")
$task.SetParameter("AssemblyFileVersion", "`$(CCNetLabel)")
 
$buildProject.Save()
$project.Save()

That Get-MSBuildProject statement is actually supplied by another NuGet package, so we’ll have to change our NuSpec file to include that as a dependency by adding to the metadata block.

<dependencies>
      <dependency id="NuGetPowerTools" version="0.26" />
    </dependencies>

It’s also a good idea to add an uninstall.ps1 script that will undo all of those changes if someone later decides not to use your package.  I’m not going to cover that here because it’s not something I’ve completed, and these packages will not be publicly posted (by all means, share your examples in the comments, if you have).

Your complete NuSpec file should now look a little bit like this

<package xmlns="http://schemas.microsoft.com/packaging/2010/07/nuspec.xsd">
  <metadata>
    <id>VersionAssembly.Mvc</id>
    <version>1.6</version>
    <authors>Kevin Blake</authors>
    <description>Version assemblies</description>
    <dependencies>
      <dependency id="NuGetPowerTools" version="0.26" />
    </dependencies>
  </metadata>
  <files>
    <file src="VersionController.cs.pp" target="content\Controllers" />
    <file src="tools\install.ps1" target="tools\install.ps1" />
</files>
</package>

It’s a very simple example, I know – but these basic steps give almost limitless possibilities to your NuGet packages… Personally, I will be creating a lot more of these to include build scripts we would otherwise have had to include in a rather large bloated project template.  NuGet+PowerShell gives the flexibility to choose just what we need from the dessert cart, and that can only be a good thing.

git out of memory errors – git repack

Saturday, January 7th, 2012

Occasionally when using git (particularly with large repositories) you might come across an error that looks a little bit like this:

remote: Counting objects: 506, done.
remote: fatal: Out of memory, malloc failed (tried to allocate 342222798 bytes)
remote: aborting due to possible repository corruption on the remote side.
fatal: protocol error: bad pack header

It means the remote server has run out of memory while packing objects on the server ready to send down the wire, and has aborted the pull request mid-way.  If you know you’ve got some particularly massive files in your repository that you didn’t really need there, you could rewrite history and get rid of them, but chances are you just need to do a little maintenance on your remote repository.

Connect to your remote server, and find the repo in question, then just run the following:

git repack -a -f -d

Now you should be able to pull your repo again, with everything already packed on the server.

Obviously, you can find more about those options with:

man git-repack

//TODO: Feature Release for v2012

Sunday, January 1st, 2012

I did one of these back in 2009 and after 2 years, I can successfully say I’ve accomplished… some of them.  It’s the sort of list I’d make an annual review if they didn’t have to follow fluffy company ideals and goals.  Not a new years resolution as such – but technologies and platforms I want to be looking at in the near future.

  • Umbraco
    • v5, in general
    • Custom v5 Hive Providers
    • Courier extensions
    • Contour extensions
  • ASP.NET MVC 4
    • Mobile Views
  • Entity Framework Code First
  • .NET 4.5
    • New async features
    • WebSockets
    • Contract-First WCF
  • T4 Templating
  • Resharper 6.1 Shared Settings
  • NuGet – more of it.  NuGet the f**k out of everything.
  • KnockoutJS
  • Gallery2/3
  • Amazon MWS.
  • Zencart, Magento, or whatever is passing as a popular choice of shopping cart nowadays

 

Google Wave – My First Bot

Sunday, November 1st, 2009

Last week, I went to the Google Technology User Group in London, which was all on the subject of Google Wave.  Lars and Steph, of Google Wave video, and Google Maps fame gave an excellent talk on Wave, how it was doing, where it was going, and the challenges they’re still facing in getting Wave ready for prime time and a public release.

I’ve had a developer account for some time now, and the talk finally got me motivated into messing around with more of the APIs.  So I created a bot.  Then I created another one.  Because the first one didn’t do anything.

1.  Get Eclipse

So far, the choices for creating Google bots are rather limited because there are rules that they must be hosted on AppEngine (for now).   So first off, get Eclipse – because it makes the entire process of doing that incredibly easy.  You can download Eclipse here.

2.  Get the AppEngine SDK.

Once you’ve got Eclipse installed and running, go to Help -> Install New Software.  Enter this URL to get at the Google AppEngine SDK.

http://dl.google.com/eclipse/plugin/3.5

3.  Create your project.

Go to File->New, and select Web Application Project.  If you don’t have that option, something’s gone wrong with your SDK download, so check step 2.

Uncheck the Googe Web Toolkit, we don’t need that.  But otherwise fill out the Project Name and Package as you see fit.

4.  Add the libraries from the Wave extensions SDK

Download wave-robots-api.jar, json.jar, and jsonrpc.jar and drop those into your project under war/WEB-INF/lib/.

Once you’ve done that, select File->Refresh, then Project->Properties from the main menu, and select Java Build Path.  Click Libraries, and Add JARs, to select the three that you’ve just added.

5.  Write your servlet class.

This is where the bulk of your bot logic (or lack of it), goes.

package helloworld;
import com.google.wave.api.*;
public class HelloWorldServlet extends AbstractRobotServlet {
	public void processEvents(RobotMessageBundle bundle) {
		for (Event e: bundle.getEvents()) {
			if (e.getType() == EventType.BLIP_SUBMITTED) {
			Blip blip = e.getBlip().createChild();
			TextView textView = blip.getDocument();
			textView.append("Hello.  Are you the world?");
			break;
			}
		}
	}
}

6. Add a servlet mapping.

Edit the file, war/WEB-INF/web.xml and add a servet-mapping just below the one you have already.

<servlet-mapping>
<servlet-name>HelloWorld<servlet-name>
<url-pattern>/_wave/robot/jsonrpc</url-pattern>
</servlet-mapping>

7. Add a capabilities file.

Add a folder under war/_wave. Create a file under that called capabilities.xml. This tells Wave which events your robot is going to respond to. In our case, we’re going to respond whenever a blip is saved (blip_submitted)./ There is a full list of capabilities in the full api docs.

<?xml version="1.0" encoding="utf-8"?>
<w:robot xmlns:w="http://wave.google.com/extensions/robots/1.0">
<w:capabilities>
<w:capability name="BLIP_SUBMITTED" content="true" />
</w:capabilities>
<w:version>3</w:version>
</w:robot>

8.  Get an AppEngine account.

That kind of finishes off the Wave-bot.  So you’ll need an AppEngine account to continue.  So go ahead and sign up, and create an application.  Incidently, I couldn’t find my created apps since I have a Google Apps / Domain account.  if that’s the case for you as well, you can find your apps list at http://appengine.google.com/a/<domain>.  Not sure why Google don’t detect that, but there we go.

9.  Deploy to AppEngine

Click the friendly little icon ae_deploy_button from your Eclipse toolbar, and enter your details to deploy you new robot.  Try not to scream “fly my pretties” as you do so.  I dare you.  Make sure you click the App Engine Project Settings button, and provide you Application ID, the same as you created in step 8.

10.  Add your app to a wave

The address of your robot will be applicationid@appspot.com, and you can add it just like any other robot.

You should now be able to interact with your bot.

bot

And if you’d like to see my first bot in action – please drop in and say hello by adding it to your wave: insulteveryone@appspot.com.

Now. World peace.  Where did I leave that file?

Rescuscitating AMM with Amazon Web Service signed requests

Wednesday, August 26th, 2009

A few days ago Amazon added a requirement to their AWS that all requests to the service be signed, lest they be rejected. I’ve been using Sozu’s excellent Amazon Media Manager plugin for a while now to manage the currently reading list on this blog. It’s been a great way to keep track of exactly what I have read (avoiding the need to, like, remember), as well as masking my illiteracy by pasting a giant list of what is commonly known as airport trash.

Unfortunately, this is one plugin that hasn’t been updated in quite a while (after all, if ain’t broke…), so it broke. Being the sort of developer that’s quite happy to pick up a block of php and hack it until it works, I stumbled across this blog entitled ‘Amazon® AWS HMAC signed request using PHP‘, which has a function to download.

So to fix AMM:

1. Download that file, and copy the contents into the bottom of amm_parser.php.

2. In the same file (amm_parser.php), replace your _setUrl function with this one:

function &_setUrl() {
	//Build URL from base URL and other required parameters
	switch ($this->_locale) {				
		case 'uk':
			$region = 'co.uk';
			break;
		case 'de':
			$region .= 'de';
			break;
		case 'jp':
			$region .= 'co.jp';
			break;
		case 'fr':
			$region .= 'fr';
			break;
		case 'ca':
			$region .= 'ca';
			break;
		case 'us':
		default:
			$region = 'com';
			break;
	}
	$public_key = "< < Your Access Key ID >>";
	$private_key = "< < Your Secret Access Key >>";
	$url = aws_signed_request($region, array(
			"Operation"=> "ItemSearch",
			"Keywords" => $this->_parameters[Keywords],
			"ResponseGroup"=>$this->_parameters[ResponseGroup],
			"SearchIndex" => $this->_parameters[SearchIndex],
			"AssociateTag" => urlencode($this->_associate_tag)),
			$public_key, $private_key);
	return $url;
}

3. Oddly enough, this new method requires a private secret key which Amazon recommends to not give to anyone. So I’m not going to post mine here, even though it’s required for the plugin to work. So before I ponder that particular nugget of madness, you’ll need to sign up for an AWS developer account, and find your own keys via the Access Identifiers page. These need to be added into the function above.

That should be enough to get you back up and running again, although selfishly I’ve only really tested it for my needs alone. So please let me know if it works, or fails miserably.

For all the legal bits, I’m not at all affiliated with Amazon or Sozu – so please use at your own risk :)