atom.xml

<?xml version="1.0" encoding="utf-8"?>
<feed xmlns="http://www.w3.org/2005/Atom">

  <title><![CDATA[My Octopress Blog]]></title>
  <link href="http://MartinWa.github.io/atom.xml" rel="self"/>
  <link href="http://MartinWa.github.io/"/>
  <updated>2013-10-08T16:00:23+02:00</updated>
  <id>http://MartinWa.github.io/</id>
  <author>
    <name><![CDATA[Your Name]]></name>
    
  </author>
  <generator uri="http://octopress.org/">Octopress</generator>

  
  <entry>
    <title type="html"><![CDATA[super-awesome]]></title>
    <link href="http://MartinWa.github.io/blog/2013/10/08/uploading-directly-to-blob-storage/"/>
    <updated>2013-10-08T14:52:00+02:00</updated>
    <id>http://MartinWa.github.io/blog/2013/10/08/uploading-directly-to-blob-storage</id>
    <content type="html"><![CDATA[<h2>Problem</h2>

<p><a href="http://firstonsite.se">Firstonsite</a>&rsquo;s goal is to be a marketplace for news media, allowing anyone to upload images and video of interesting events and offer it news desks for sale. To be able to handle large amounts of media files from many concurrent users we wanted to leverage the power of <a href="http://www.windowsazure.com/sv-se/">Windows Azure</a> and its Blob Storage to allow the users to upload directly into our blob storage without first having to pass through our web back-end. This allow us to avoid a bottleneck when several users wants to upload large files and remove the need for creating several web roles just for this purpose.</p>

<p>To accomplish this we needed the new new <a href="http://www.w3.org/TR/FileAPI/">File API</a> that comes in HTML 5 that allows JavaScript to handle file upload without having to touch the backend. The Firstonsite project have previously decided to require the user to have a HTML 5 compliant browser, using the &lt;video&gt; tag for instance, so this was not a problem.</p>

<h2>Solution</h2>

<p>When we first tried to implement this solution we ran into an unexpected problem, Blob Storage does not have <a href="http://en.wikipedia.org/wiki/Cross-origin_resource_sharing">CORS</a> support (<a href="http://feedback.windowsazure.com/forums/217298-storage/suggestions/2850796-support-cross-origin-resource-sharing-cors-via-a?page=1&amp;per_page=20">yet</a>).</p>

<p>We decided to push the feature until CORS was supported but recently I found a very interesting <a href="http://gauravmantri.com/2013/02/16/uploading-large-files-in-windows-azure-blob-storage-using-shared-access-signature-html-and-javascript/">blog by Gaurav Mantri</a> with a workaround of the problem. The workaround is to place the code for the file upload on the Blob Storage thus removing the need for cross domain access.</p>

<h2>Example code</h2>

<p>I have created an example site implementing Mantri&rsquo;s solution using a controller to create a <a href="http://msdn.microsoft.com/en-us/library/windowsazure/jj721951.aspx">Shared Access Signature</a> and sending this to the static HTML page on the Blob Storage. In order to prevent people from draining my Azure account dry I used the MVC 4 Simple Membership provider template to add login to the page.</p>

<p>The upload controller creates the SAS and adds it as a query parameter to the address of the static site. To avoid Blob Storage parsing the parameter (assuming that it is a SAS used to access the requested HTML file) the string is base 64 encoded first.</p>

<p>The upload.js script running at the Blob Storage extracts the sas query parameter using <a href="https://github.com/allmarkedup/purl">purl.js</a> and decodes the string to get the URI for the file to be uploaded to.</p>

<p>The user can select a file and when the <strong>Upload File</strong> button is pressed a FileReader starts reading slices of that file. Each slice is uploaded to the Blob Storage together with a six digit block id. This Id is later used to combine the slices at the Blob Storage.</p>

<p>If an error occurs the code will retry the transmission of the current slice several times before failing the file upload. This takes care of occasional glitches that can occur normally.</p>

<p>After all slices have been uploaded a list of all slices and their order are sent as a block list and the file transfer will be complete.</p>

<p>All code can be found at <a href="https://github.com/MartinWa/BlobUploader">https://github.com/MartinWa/BlobUploader</a></p>

<h2>How to test it</h2>

<p>The easiest way to test this is to clone the code from GitHub to a local folder (using git), open the solution file in Visual Sudio 2012 (with the Azure SDK), start the storage emulator and run the debugger.</p>

<p>The contents of the blobstorageFiles folder must be stored on the development Blob Storage in a container named public (with public access rights). Note that the content type of the HTML file must be set to text/html. I do all this using the <a href="http://azurestorageexplorer.codeplex.com/">Azure Storage Explorer tool</a>. If you have done everything correct you should be able to browse to <a href="http://127.0.0.1:10000/devstoreaccount1/public/upload.html.">http://127.0.0.1:10000/devstoreaccount1/public/upload.html.</a></p>

<p>In order to be able to access the Upload tab you must create a uploader entry in the webpages_Roles table and then set that role to your user in webpages_UsersInRoles.</p>

<h2>Deploy your own</h2>

<p>If you like to deploy to your own Windows Azure Web Site clone the repository to your own GitHub account.
Create a new Azure Web Site using the custom create wizard
<img src="https://dynablog.s3.amazonaws.com/uploads/article_image/image/10/CreateCustomWebsite.png" alt="Custom Create Web Site" /></p>

<p>adding your GitHub repository
<img src="https://dynablog.s3.amazonaws.com/uploads/article_image/image/11/IntegrateGitHub.png" alt="Integrate GitHub" /></p>

<p>Now the management portal will deploy the main branch of the repository to the Web Site. It will also deploy any changes you push to main giving you an instant continuous deployment.</p>

<p>Since we need file storage we also need to create a storage account
<img src="https://dynablog.s3.amazonaws.com/uploads/article_image/image/12/CreateStorage.png" alt="Create Storage Account" /></p>

<p>Now all that remains is to set the database connection and the storage account. Go to configure on the Web Site and add an app setting:
<code>StorageConnection : DefaultEndpointsProtocol=https;AccountName=xxx;AccountKey=<strong><em></code> replacing the account name with the name of your storage account and </em></strong> with the primary access key. Then rename the <strong>DefaultConnection</strong> connection string to <strong>DatabaseConnection</strong></p>

<p>The site should now be up. As in the local case you need to add roles and copy the static HTML page to the Blob Storage (set up the account in the Windows Azure explorer).</p>

<p>See my site at <a href="http://blobupload.azurewebsites.net/">http://blobupload.azurewebsites.net/</a></p>

<h2>Future improvements</h2>

<p>The file upload should be done in parallel to improve upload speed..</p>
]]></content>
  </entry>
  
  <entry>
    <title type="html"><![CDATA[Zombie Ninjas Attack: A survivor's retrospective]]></title>
    <link href="http://MartinWa.github.io/blog/2013/10/08/zombie-ninjas-attack-a-survivors-retrospective/"/>
    <updated>2013-10-08T14:46:00+02:00</updated>
    <id>http://MartinWa.github.io/blog/2013/10/08/zombie-ninjas-attack-a-survivors-retrospective</id>
    <content type="html"><![CDATA[
]]></content>
  </entry>
  
  <entry>
    <title type="html"><![CDATA[First]]></title>
    <link href="http://MartinWa.github.io/blog/2013/10/08/first/"/>
    <updated>2013-10-08T14:45:00+02:00</updated>
    <id>http://MartinWa.github.io/blog/2013/10/08/first</id>
    <content type="html"><![CDATA[
]]></content>
  </entry>
  
</feed>