Using Umbraco with Web Application projects, CI, across teams

January 28th, 2012

Can I use just a Web Site project?

Yes – of course.

I don’t really want to get into the debate about Web Site projects, vs. Web Application projects.  But I do want to start off by saying I dislike Web Site projects, and we’d never think about using them anywhere else.

One thing they offer though, is all of the files you drop into the directory become part of your Visual Studio solution.  This is really useful for applications such as Umbraco where developers are uploading media items, adding scripts, or creating templates via the Umbraco back-end.  It’s a pain to have to switch back to Visual Studio and add those generated files to your project, just to ensure they get published with the rest of your site.

We use a CI server, we want repeatable deployments, we’re working in teams, we want everything that’s anything to be in source control, we want the moon on a bloody stick – and Web Site projects just don’t cut it.  But I don’t really want to even mess around with everyone having to only work in Visual Studio either, so PostBuild events on separate projects that copy in razor scripts / templates etc is also – no good.  The beautiful thing with Umbraco is it’s completely flexible, there’s no right or wrong way to use it.  So here’s how we’re (currently) managing Umbraco (v4) builds.

Getting started with your web application

Firstly, create a new Empty Web application project in Visual Studio.

Download and copy all of the files from the latest Umbraco release into your project.

Don't forget the show all files button, to see everything in your directory

Now in Visual Studio, you want to add all the directories / files to your solution that you aren’t going to let people manage through Umbraco as well.  For our purposes, that’s config, data, App_Data, App_Browsers, data, umbraco, umbraco_client, default.aspx, and web.config.  If it’s still useful to have some of those other files/directories in your project later, don’t worry – you can add them (and have Umbraco/msbuild manage the rest).

You can add the bin folder as well, if you want – but we don’t like all binary files checked into sourcecontrol, so this is one folder we usually exclude by default.

If you want the same, copy all of the files out of the bin folder to a directory outside of your csproj (call it Resources, Libraries, whatever you like).  Then in Visual Studio, Right-click on your references folder, Add reference, and include all those dlls.  You’ll need that Resources / Libraries folder included in sourcecontrol as well.

You now have a solution that will build and work locally.  But you still don’t have the ease of including new Umbraco-managed items in your builds (try and right-click to publish your project to the file system, and notice how you’re still missing all of your media items and masterpages).  That’s no good for our Continuous Integration server, and a Web Site project would include all those items by default (you’d need to create one of those Web Deployment project as well though).

Modify MsBuild scripts… Include everything the wrong way

Msbuild to the rescue.  Open your .csproj file in your favourite editor (unload the project in Visual Studio if you want to use that).

You’ll see a whole bunch of xml nodes like <Content Include=”default.aspx”>.  These tell msbuild to include these files as Content items, and copy them to the Output directory on every build.

You can use wildcards, so you could just add a new item such as <Content Include=”masterpages\**\*.master” /> and you’d think that would include everything.  And you’d be right – reload your solution in Visual Studio,  and all of those masterpages will be included.  However – exclude one file, re-add it, and save your project.  If you inspect your .csproj file again – look how helpful Visual Studio has been… It’s replaced your wildcard map with a list of individual files again.  Bad Visual Studio.

Modify MsBuild scripts… Include everything the right way

Since that’s clearly going to go wrong somewhere down the line, one of your developers is going to do exactly that and unknowingly bring your entire build solution crumbling to its knees – this clearly won’t do.  There is always another way.

We can modify the BeforeBuild event (that’s currently commented out near the bottom of your csproj file, so uncomment it and replace it with this.

<Target Name="BeforeBuild">
<CreateItem Include="macroScripts\*.cshtml">
<Output ItemName="Content" TaskParameter="Include" />
</CreateItem>
<CreateItem Include="xslt\*.xslt">
<Output ItemName="Content" TaskParameter="Include" />
</CreateItem>
<CreateItem Include="masterpages\**\*.master">
<Output ItemName="Content" TaskParameter="Include" />
</CreateItem>
<CreateItem Include="media\**\*.*">
<Output ItemName="Content" TaskParameter="Include" />
</CreateItem>
<CreateItem Include="usercontrols\**\*.*">
<Output ItemName="Content" TaskParameter="Include" />
</CreateItem>
<CreateItem Include="css\**\*.*">
<Output ItemName="Content" TaskParameter="Include" />
</CreateItem>
</Target>

What we’re doing is adding to the Content array that msbuild’s defined with all those Content nodes.  We’re including all files from those directories in the publish, and we’re making the Web Application project work the way that Umbraco, and we want.  You can include as many of the folders as you’re comfortable with, and you can continue to work in Visual Studio for as much or as little of the sln as you want.

Avoid doing this next time

Having putting in this small amount of work up-front, you can now export your entire project as a project template, and use it to start off all your projects in the future.  You probably have more of a white label sln you can add to this and make your subsequent builds even easier.  I won’t get into the details of that in this post (but it’s essentially just File-Export Template within Visual Studio).

Like I said, Umbraco is completely flexible in the way you approach your builds, this is one solution that works for us – and there is always another way.  I’d be keen to hear your feedback / improvements / alternatives – we haven’t etched it in stone by any means.

Similar posts that have inspired, and Another Way:

(Send me more, and I’ll add to this list)

git out of memory errors – git repack

January 7th, 2012

Occasionally when using git (particularly with large repositories) you might come across an error that looks a little bit like this:

remote: Counting objects: 506, done.
remote: fatal: Out of memory, malloc failed (tried to allocate 342222798 bytes)
remote: aborting due to possible repository corruption on the remote side.
fatal: protocol error: bad pack header

It means the remote server has run out of memory while packing objects on the server ready to send down the wire, and has aborted the pull request mid-way.  If you know you’ve got some particularly massive files in your repository that you didn’t really need there, you could rewrite history and get rid of them, but chances are you just need to do a little maintenance on your remote repository.

Connect to your remote server, and find the repo in question, then just run the following:

git repack -a -f -d

Now you should be able to pull your repo again, with everything already packed on the server.

Obviously, you can find more about those options with:

man git-repack

//TODO: Feature Release for v2012

January 1st, 2012

I did one of these back in 2009 and after 2 years, I can successfully say I’ve accomplished… some of them.  It’s the sort of list I’d make an annual review if they didn’t have to follow fluffy company ideals and goals.  Not a new years resolution as such – but technologies and platforms I want to be looking at in the near future.

  • Umbraco
    • v5, in general
    • Custom v5 Hive Providers
    • Courier extensions
    • Contour extensions
  • ASP.NET MVC 4
    • Mobile Views
  • Entity Framework Code First
  • .NET 4.5
    • New async features
    • WebSockets
    • Contract-First WCF
  • T4 Templating
  • Resharper 6.1 Shared Settings
  • NuGet – more of it.  NuGet the f**k out of everything.
  • KnockoutJS
  • Gallery2/3
  • Amazon MWS.
  • Zencart, Magento, or whatever is passing as a popular choice of shopping cart nowadays

 

Farewell 2011, Long Live 2012

January 1st, 2012

I’ve seen tons of 2011 retrospective blog posts over the past month. Not having one is making me feel inadequate, so gathered from various stats on my machine, and social networks – last year, I:

My most-listened to artists from 2011

  • Read 18 novels.
  • Listened to 17,650 tracks.
  • Drank 102 beers (88 unique) since August… That I checked into.
  • Wrote 3 blog posts (must try harder next year – already a third of the way there).
  • Visited 3 new countries – grand total distance between the capitals and London of approx. 26,658km.
  • Took approx. 7,597 photos.
  • Uploaded 1,249 photos to Blakepics.
  • Recorded 667 TV shows with MythTV, totalling 20 days, 21 hours of viewing.
  • Pinned 94 pictures on Pinterest.
  • Tweeted 553 times.
  • Played 28 xbox games
  • Shared far too much information with social networks, and online services.

What have I missed?

Bulgaria Photo Story

October 23rd, 2011

I’ve just been back out to The Balkans again, this time to Bulgaria… After heading out to SE Asia a few times in the last year or two, it’s really nice to return my travelling to my favourite part of Europe. This area offers a real melting pot of culture, history, people, and beautiful landscapes, without the bustle of more well-worn destinations elsewhere in the world – and Bulgaria is no exception. But don’t go there – I like things the way they are :)

The music is “More” by Tyrone Wells.

You can also find these photos elsewhere online