web 2.0

Manipulating Blob Data in MSSQL with C#

Storing BLOB (Binary Large Objects) in a SQL database can be a very convenient way to tie documents with metadata. For example, if you are building a document management system it is very nice to be able to stuff a word document in a data row along with other information such as who created the document, when it was modified and etcetera.

The first step in accomplishing this task is to define a database table. The only real trick is to use a image column for the datatype of the column storing the BLOB data. For example consider this table creation script:

CREATE TABLE [dbo].[PurchaseOrderAttachment](
    [ID] [int] IDENTITY(1,1) NOT NULL,
    [PurchaseOrder] [nvarchar](50) NOT NULL,
    [Data] [image] NOT NULL,
    [Filename] [nvarchar](100) NOT NULL
)

To insert the data using C# you will need the following code (using LINQ)

MyDataContext db = new MyDataContext();
PurchaseOrderAttachment a = new PurchaseOrderAttachment();
a.Filename = System.IO.Path.GetFileName(attachment.Filename);
a.Data = System.IO.File.ReadAllBytes(attachment.Filename);
a.PurchaseOrder = "123";
db.PurchaseOrderAttachments.Add(a);
db.SubmitChanges();

Or if you are old school and want to do it with regular T-SQL:

INSERT INTO PurchaseOrderAttachment
    ( PurchaseOrder , Data, Filename )
VALUES
   ('123'
    ,( SELECT *  FROM OPENROWSET(BULK 'c:\windows\Santa Fe Stucco.bmp', SINGLE_BLOB) 
as x) ,'Santa Fe Stucco.bmp')

After we insert the image into the database we can take a quick look with SQL Server Management studio to see what it looks like in the table:

You can see that the Data column has binary data in it and it is not readable with the human eye. Therefore in order to view the data you will need to write a little code. Fortunately, for you I have written this code in two different ways. The first implementation was for a WinForms application. The second implementation was from a MVC application. Here is the Winforms code:

   1:  PurchaseOrderAttachment attachment = FindAttachment(1);
   2:  string filename = Path.Combine( System.IO.Path.GetTempPath(), attachment.Filename );
   3:  FileStream fs = new FileStream(filename, FileMode.Create, FileAccess.Write);            
   4:  fs.Write(attachment.Data.ToArray(), 0, attachment.Data.Length);
   5:  fs.Close();
   6:   
   7:  if (File.Exists(filename) == true)
   8:      System.Diagnostics.Process.Start(filename);
   9:  else
  10:      MessageBox.Show(String.Format("Could not retreive file \"{0}\"", filename),
  11:                            "File not found", MessageBoxButtons.OK, MessageBoxIcon.Information);

Lets dissect this code. Line 1 retrieves the record from the database. Line 2 creates a temporary place to store the file on disk. Line 3,4 and 5 saves the file to disk using the FileStream object. Then on Line 8 I call System.Diagnostics.Process.Start(filename) to open the file. The reason I used this technique was because you can store any kind of file in the database. It could be excel, word, a jpeg, a TSQL script or even a text file. By saving the file to disk and calling the Process.Start method on it I can rely on windows to use the file extension and determine which program to launch. However, I can eliminate the need to write the file to disk completely if I build some more intelligence into my code and decide which application to launch based on the extension or content of the file. This is demonstrated in my next example.

In my MVC Web Application, I limited the kinds of files I allowed for upload. I stream the file to the client by using an HTTP Handler:

   1:  <%@ WebHandler Language="C#" Class="POAttachment" %>
   2:   
   3:  using System;
   4:  using System.Web;
   5:  using DBA.Common.DAL;
   6:  using System.Data.Linq;
   7:  using System.Linq;
   8:  using System.IO;
   9:   
  10:  public class POAttachment : IHttpHandler {
  11:      
  12:      public void ProcessRequest (HttpContext context) {
  13:          try {
  14:              int id = Convert.ToInt32(context.Request["id"]);
  15:              DBAInventoryDataContext db = new DBAInventoryDataContext();
  16:              PurchaseOrderAttachment attachment = FindAttachment(id);
  17:   
  18:              FileInfo file = new FileInfo(attachment.Filename);
  19:              context.Response.ContentType = GetContentType(file.Extension);
  20:              context.Response.AddHeader("Content-Disposition", " filename=" + attachment.Filename);
  21:              context.Response.AddHeader("Content-Length", attachment.Data.Length.ToString());
  22:              context.Response.BinaryWrite(attachment.Data.ToArray());
  23:              context.Response.Flush();            
  24:          }
  25:          catch {
  26:              context.Response.ContentType = "text/plain";
  27:              context.Response.Write("Error retreving attachment");
  28:          }
  29:      }
  30:      
  31:      private string GetContentType(string fileExtension)
  32:      {
  33:           switch (fileExtension) {
  34:               case ".xls":
  35:                   return "application/vnd.ms-excel";           
  38:               case ".pdf":
  39:                   return "application/pdf";             
  40:               default:
  41:                   return "application/octet-stream";
  42:           }
  43:      }
  44:  }

This code is somewhat simple. On line 16 I retrieve the row from the database. Then on lines 19-23 I set the Content Type, add some required headers and then I write the contents of the Data field to the Response object. When I want to use this Handler I create a hyperlink in my application that looks like this: http://<mysite>/POAttachment.ashx?id=1. When the user clicks on the link the handler will fetches the data from the database. The browser then will then launch the appropriate application based on the Content Type set in the Response headers.

Hopefully, this code will save you a little time. Although, it is very simple when you see the final code it takes a little bit of reading and experimenting to get something working for the first time. Happy Coding!

VS 2010 Intellisense Improvements

One of the reasons why it is so enjoyable to write .NET code is the IDE. If you started out writing code in notepad (like me) then I am sure that you truly appreciate all of the amazing things that Visual Studio does. Even though VS 2008 is amazing, VS 2010 is even better. One of the things they are upgrading for the next version is intellisense.

In Visual Studio 2008, when you typed the first couple characters or a property or method, intellisense only returned items that started with those characters. In Visual Studio 2010 when you start typing text you get all the properties and methods start with and also contain those characters. For example, If you type "grid.Edit". You will not only get the method EditIndex (as expected) but you will also get RowCancelingEdit, SetEditRow as well. The method EditIndex will be the default selection because it is a obvious match but you will also see the wildcard matches listed as well.

One other new feature has been titled Pascal Case Intellisense. Since the default naming guidelines in .NET use pascal casing, each word in a type or member should start with a capitalized letter. VS 2010's intellisense filtering support now enables you to take advantage of this to quickly find and filter methods.  For example, if we typed grid.PIC VS 2010 will filter results that have PIC in their name, as well as those members which have a pascal cased name where the word segments start with that letter sequence. In our example, we would get two results which are PageIndexChanged and PageIndexChanging.

Additional Resources:

How To Optimize the Speed of Your Website

With all the fancy scripts and tools available these days for web development it is easy to get carried away. Instead of writing custom solutions, we tend to download large, multi-purpose scripts to accomplish our goals. Although each library may not be big by itself, when you keep stacking one on top of the other you can end up with a bloated website. By bloated, I mean that your pages will become large in size which will increase download time and ultimately affect the user experience. So here are some tips that I came up with to help streamline your website.

  1. Include only the relevant scripts: If you are using ASP.NET, put a ContentPlaceHolder tag in the head of your master page which can be used for additional scripts and CSS. Instead of including every possible script file and CSS file in the master page you can now optionally include them on the pages that really need it.
    <head runat="server">       
        <link href="/Content/site.css" rel="stylesheet" type="text/css" />   
        <script src="/Scripts/jquery-1.3.2.min.js" type="text/javascript"></script>                  
         <asp:ContentPlaceHolder ID="AdditionalScripts" runat="server" />   
    </head>
    The HEAD on my master page is very simple. When I need to use additional libraries like JQuery UI, I include them on the content pages instead by using the ContentPlaceHolder.
  2. Optimize the size of your JavaScript and CSS: Removing the extra whitespace in your CSS and JavaScript files will decrease the size. There are some handy online tools you can use to accomplish this task.
    • The Online CSS compressor which offers three levels of compression which are Light, Normal and Super Compact. In addition it will also strip out comments.
    • The Online JavaScript compressor also offers three levels of compression. In my testing I was able to take a 25 KB script file and shrink it down to 5KB using the aggressive compression level.
    When you compress your files make sure you keep a copy of your original script and CSS files. Because, when you debug and develop it will be easier to read the files with the extra whitespace and comments.
  3. Be careful with images: A picture may paint a thousand words but it also can consume thousands of kilobytes! If you use images, try to follow these simple rules:
    • Use images sparingly. Ask yourself, do I really need to have an custom image for a button or can I use the standard HTML buttons with a little CSS styling.
    • Don't take a 800x600 dimension image and set the width and height on it to turn it into a icon. Shrink the image instead using Photoshop or GIMP.
    • Use small images for backgrounds instead of large full screen images. Often times, you can just tile the small image using CSS and accomplish the same goal.
    • Pick the proper image format. Sometimes just converting an image from one format to another can give you a large savings. Also reducing the resolution of the image can help. In most cases, you won't even notice the difference. Look at the example below from pingdom

  4. Write smarter code: Often times you can shrink your website just by writing smarter code. Take this CSS class for example:
    .myClass { border-left: 0px; border-right: solid 1px #fff; border-top: solid 1px #ff, border-bottom: solid 1px #fff }

    Can be rewritten as:
    .myClass { border: solid 1px #fff; border-left: 0px; }

    For more tips like this, visit the CSS shorthand Guide...
  5. Consider using HTTP compression: Finally, if you followed all of these tips and your webpage is still too big then consider turning on compression:
    1. For IIS 6.0 follow this guide 
    2. For IIS 7.0 follow this guide.
    3. And of course, if you are using Apache follow these instructions.

Just keep in mind that turning on compression can increase the CPU utilization on your web server. So make sure you test out compression first to ensure that your web server can handle the increased load.

Export to CSV using Reflection and IQueryable

Whenever I build an application that contains any kind of statistical data I commonly get the request to make all of the grids and/or tables to include a export to CSV capability. In my mind, there are two options to approach this functionality.

  1. Build a method that iterates over a control. For example in a grid you can iterate over the columns and rows to build the CSV file.
  2. Build a method that iterates over the data.

In my opinion approach #2 is more flexible and lends itself better to code re-use. Tying export functionality to a specific control is really not a good idea unless you have a unique situation which requires it.

In my MVC application I am using LINQ to SQL and I used the Repository pattern that you may have seen before if you have ever looked at the nerddinner.com source code. The majority of my data access code returns an IQueryable<T> result. Therefore my export function is an extension method on the IQueryable<T> type. In order to get the column names and values I use reflection and append the values to a CSV file. Here is the snippet of code that prints the header row of the CSV file...

foreach (var obj in list) {
   Type type = obj.GetType();
   PropertyInfo[] properties = type.GetProperties();
 
   string s = String.Empty;
   if ( !headerPrinted  ) {
      foreach (PropertyInfo propertyInfo in properties) {
         if (exclude.Count(x => x.Contains(propertyInfo.Name)) > 0)
            continue;
 
         if (s.Length > 0)
            s = String.Join(",", new string[] { s, propertyInfo.Name });
         else
            s = propertyInfo.Name;
      }
      sw.WriteLine(s);
      headerPrinted = true;
   }

The list variable is the IQueryable<T> object that was past into the method. For each object in the list I call GetType() to find out what kind of object it is and then I call GetProperties() on it. This allows me to enumerate over the properties and get the name of each property so I can write it to the CSV file. You may also notice that I added the ability to exclude certain columns  as well. This is because that many of my Repository methods include extra columns which are used to build action links in my views. For example, If I am displaying information about a person in a grid I will commonly include their full name and the primary key of the record in the result set so I can produce a hyperlink that the end user can click on to view the details. In MVC, we would use the ActionLink HtmlHelper to create the hyperlink...

<%= Html.ActionLink(contact.FullName, "Details", new { ID= contact.ID })%>

The action link call above would produce the following HTML:

<a href="/Contact/Details/1">John Smith</a>

However, when I export the records to CSV, the primary key is generally not needed so I can exclude this from the export by specifying the "ContactID" in the exclude parameter. This allows me to re-use methods in my repository class. Otherwise, I would have to write special methods which excluded the key fields for exporting purposes.

Finally, when you look at the completed method below you will probably notice that at the very end of the method I added some web specific code. This code sets the content type and response headers so the CSV file can be viewed with Excel in a browser. If you want to make the method generic enough so it can be used on the Web and/or WinForms you can extract this section of code and put it in a separate method.

public static void ExportToCSV<T>(this IQueryable<T> list, string filename, string[] exclude ) {
    StringWriter sw = new StringWriter();
    bool headerPrinted = false;
    foreach (var obj in list) {
        Type type = obj.GetType();
        PropertyInfo[] properties = type.GetProperties();

        string s = String.Empty;
        if ( !headerPrinted  ) {
            foreach (PropertyInfo propertyInfo in properties) {
                if (exclude.Count(x => x.Contains(propertyInfo.Name)) > 0)
                    continue;

                if (s.Length > 0)
                    s = String.Join(",", new string[] { s, propertyInfo.Name });
                else
                    s = propertyInfo.Name;
            }
            sw.WriteLine(s);
            headerPrinted = true;
        }

        s = String.Empty;                                
        for (int idx = 0; idx < properties.Length; idx++) {

            if (exclude.Count(x => x.Contains(properties[idx].Name)) > 0)
                continue;

            var value = properties[idx].GetValue(obj, null);
            var formattedValue = value == null ? String.Empty : value.ToString();
            
            if (value != null) {
                if (value.GetType() == typeof(string))
                    formattedValue = "\"" + formattedValue + "\"";
            }

            if (s.Length > 0)
                s = String.Join(",", new string[] { s, formattedValue });
            else
                s = formattedValue;

        }
        sw.WriteLine(s);
    }
    HttpResponse response = System.Web.HttpContext.Current.Response;
    response.AddHeader("Content-Disposition", "attachment; filename=" + filename ); 
    response.ContentType = "application/ms-excel"; 
    response.ContentEncoding = System.Text.Encoding.GetEncoding("utf-8"); 
    response.Write(sw); 
    response.End();             
}

If you are going to use this code in a web application you may also be interested in this HTML code. It basically renders a select box and a button. When the button is clicked it opens a new window using the URL of the selected option. In this example, the controller action will fire my ExportToCSV routine which will create the CSV file and display it in a new browser window.

<span style="float:right;">
   <b>Export:</b>
   <select id="export">         
      <option value="/Instance/ExportAssignments">CSV</option>
   </select>
   <input type="button" value="Go" onclick="window.open( $('#export').val() )" />
</span>

I chose to use a drop down list here because I intend to add other export formats later on such as HTML and PDF. I will also probably create a HTMLHelper class to render the "export" HTML in my views. This way I can have a consistent and centrally maintainable export control which I can use across my entire MVC web application.

Additional Resouces:

Doing AJAX the easy way with JQuery and MVC

I know I have been writing a lot about MVC and JQuery lately. What can I say? JQuery makes writing JavaScript fun! Admittedly, when I first start using it, it was a little overwhelming. I was confused by the dollar sign notation and I really did not understand why it was better than plain old JavaScript. However, now that I am getting better acquainted with JQuery I can not imagine life without it.

Recently I have been busy converting a Windows Form app into a MVC application. Unfortunately, when people get used to the rich user experience of a windows application they tend to have trouble adjusting to the stateless and often less extravagant experience of a web application. Most web apps function by making round trips to the server, which results in a full page refresh to display the data. This is annoying when you are scrolled halfway down a page looking at a record and the page refreshes and shoots you back to the top again.

In order to avoid unnecessary post backs we can use AJAX (asynchronous JavaScript and XML). To simplify the process of writing the JavaScript we can leverage the power of JQuery's post function. Here is an example:

The method in the controller: Notice that it takes a single parameter and returns a string. The details of the implementation aren't important.

public string JulianToDate(int julian) {
    return DBA.Core.DateTimeUtilities.JulianToDateTime(julian).ToString("MM/dd/yyyy");
}

The Javascript: We use the post method from the jquery library to call the method in our controller. The first parameter is the URL of the method, the second parameter are the values we pass to the method and the final parameter is the callback method.

function RunJulianToDate() {
   var input = $("input#julian1").val();
   $.post( "/Utils/JulianToDate", 
           { julian : input },
           function ( data ) {
               $("input#date1").val(data);
   });         
}

Notice in the callback function that a variable is returned. In this simple example it is just a plain old string that we use to populate a textbox with. However, we can do other things like return a JSON (JavaScript object notation) string, deserialize it using JQuery and dynamically update a table or HTML element. For example, If you wanted to return a JSON from your method, you would instantiate an object and serialize by using the Serialize<T>() method of the System.Web.Script.Serialization.JavaScriptSerializer class. Here is an example:

public String GetPurchaseOrderJson(string id) {
   PurchaseOrderModel po = repository.FindPurchaseOrder(id);
   return new JavaScriptSerializer().Serialize(po);
}

In your JavaScript code you can deserialize the JSON string into an object by using the eval method:

function GetPurchaseOrder() {
   $.post( "/Utils/GetPurchaseOrderJson", 
           { id : 1 },
           function ( data ) {
               eval( "var po = " + data );
            alert( po.PurchaseDate );
});         
}