Thursday, March 24, 2011

A LogParser Input Plugin to Search an SVN Repository

Summary

Below is an example of how to write a custom LogParser input plugin, using SharpSvn to query an svn repository.

Download the LogParser.Svn project here. You will need Visual Studio 2010 Express to build it. Make sure that the build tab in the project properties has "Register for COM interop" checked. You will also need to download the latest build of SharpSvn and update the references in the project.

Any reference to examples refer to the following command line:

LogParser.exe -rtp:-1 -i:COM -iProgID:LogParser.Svn.SvnLogParserInputContext -iCOMParams:startDate=2010-01-01 "Select Count(1), Author From http://my.svn/repo/trunk Group By Author Order By Count(1)"

To start, create a new "Class Library" project in Visual Studio. I've called mine LogParser.Svn.

Next, we need to create copies of the COM interface and enum our code will use:

    public enum FieldType : int
    {
        Integer = 1,
        Real = 2,
        String = 3,
        Timestamp = 4,
        Null = 5,
    };

    public interface ILogParserInputContext
    {
        void OpenInput(string from);
        int GetFieldCount();
        string GetFieldName(int index);
        FieldType GetFieldType(int index);
        bool ReadRecord();
        object GetValue(int index);
        void CloseInput(bool abort);
    }

Now we need a class that implements the interface defined above. Let's start with the OpenInput method. This method will receive a string that is entered in the query given to LogParser that represents the "FROM" statement. Taking the command line above, it would receive a value of "http://my.svn/repo/trunk".

        public void OpenInput(string from)
        {
//            System.Diagnostics.Debugger.Launch();

            _client = new SvnClient();
            _repository = from;

            SvnLogArgs args = new SvnLogArgs();

            if (_start.HasValue)
            {
                if (_end.HasValue)
                {
                    args.Range = new SvnRevisionRange(_start.Value, _end.Value);
                }
                else
                {
                    args.Range = new SvnRevisionRange(new SvnRevision(_start.Value), SvnRevision.Head);
                }
            }
            else if (_end.HasValue)
            {
                args.Range = new SvnRevisionRange(SvnRevision.Zero, new SvnRevision(_end.Value));
            }

            if (_startDate.HasValue)
            {
                if (_endDate.HasValue)
                {
                    args.Range = new SvnRevisionRange(_startDate.Value, _endDate.Value);
                }
                else
                {
                    args.Range = new SvnRevisionRange(new SvnRevision(_startDate.Value), SvnRevision.Head);
                }
            }
            else if (_endDate.HasValue)
            {
                args.Range = new SvnRevisionRange(SvnRevision.Zero, new SvnRevision(_endDate.Value));
            }

            _results = new List();

            _client.Log(new Uri(_repository), args, delegate(object sender, SvnLogEventArgs e)
            {
                if (_splitFiles)
                {
                    foreach (SvnChangeItem i in e.ChangedPaths)
                    {
                        object[] result = new object[6];

                        result[0] = e.Author;
                        result[1] = e.LogMessage;
                        result[2] = e.Revision;
                        result[3] = e.Time;
                        result[4] = i.Path;

                        _results.Add(result);
                    }
                }
                else
                {
                    object[] result = new object[6];

                    result[0] = e.Author;
                    result[1] = e.LogMessage;
                    result[2] = e.Revision;
                    result[3] = e.Time;

                    StringBuilder paths = new StringBuilder();

                    foreach (SvnChangeItem i in e.ChangedPaths)
                    {
                        paths.AppendFormat("{0};", i.Path);
                    }

                    result[4] = paths.ToString();

                    _results.Add(result);
                }

            });
        }

First, the code above creates a new SvnClient object, which we will us to view log messages. Next we create an SvnLogArgs object, and set the revision range we want. Finally, we call SvnClient.Log. In the delegate, we receive each log entry matching our SvnLogArgs object. We then create an array of values and store them in a List. Ideally, we would only prepare to receive a value, and then receive each entry during each call to ReadRecord, but I haven't gotten around to doing that (but it shouldn't be too difficult to change the code to work this way).

The next few methods are very straight forward:

        public int GetFieldCount()
        {
            return 5;
        }

        public string GetFieldName(int index)
        {
            switch (index)
            {
                case 0:
                    return "Author";
                case 1:
                    return "LogMessage";
                case 2:
                    return "Revision";
                case 3:
                    return "Time";
                case 4:
                    return "ChangedPaths";
            }

            return null;
        }

        public FieldType GetFieldType(int index)
        {
            switch (index)
            {
                case 0:
                    return FieldType.String;
                case 1:
                    return FieldType.String;
                case 2:
                    return FieldType.Integer;
                case 3:
                    return FieldType.Timestamp;
                case 4:
                    return FieldType.String;
            }

            return FieldType.Null;
        }

        public bool ReadRecord()
        {
            return ++_rowNumber < _results.Count;
        }

        public object GetValue(int index)
        {
            return _results[_rowNumber][index];
        }

        public void CloseInput(bool abort)
        {
            _client.Dispose();
        }

GetFieldCount returns how many fields your parser can return. GetFieldName returns what the field name is for a given index. GetFieldType returns the type of field for a given index. ReadRecord is where your plugin should advance to the next record, and return true if another record is found. GetValue returns the value of the specified field for the current record. CloseInput is called at the end, and is where you should do any cleanup needed.

You can also create some write-only properties that can be set on the command line using -iCOMParams:prop=value. See the attached code for an example.

I hope this helps people trying to start using SharpSvn or writing their own LogParser input plugin.

Thursday, March 10, 2011

.Net 4.0 MemoryCache with SqlChangeMonitor

Summary

There isn't a lot of documentation on the internet about how to use the SqlChangeMonitor with the new MemoryCache class in .Net 4.0, so I thought I would add my example:

Database Preparation

The first step is to prepare your database for SqlChangeMonitor. This feature uses the SQL Server Service Broker to setup a notification event that fires to notify when data changes that would change the returned recordset of a query, so we have to enable service broker on your database:

ALTER DATABASE database_name SET TRUSTWORTHY ON WITH ROLLBACK IMMEDIATE
ALTER DATABASE database_name SET ENABLE_BROKER WITH ROLLBACK IMMEDIATE
ALTER AUTHORIZATION ON DATABASE::database_name TO sa

With that out of the way, we can continue on to setting up the cache in code…

Code

public bool IsInMaintenanceMode()
{
 bool inMaintenanceMode;

 if (MemoryCache.Default["MaintenanceMode"] == null)
 {
  CacheItemPolicy policy = new CacheItemPolicy();

  string connStr = "MY CONNECTION STRING";

  SqlDependency.Start(connStr);

  using (SqlConnection conn = new SqlConnection(connStr))
  {
   using (SqlCommand command = new SqlCommand("Select MaintenanceMode From dbo.Maintenance", conn))
   {
    command.Notification = null;

    SqlDependency dep = new SqlDependency();

    dep.AddCommandDependency(command);

    conn.Open();

    inMaintenanceMode = (bool)command.ExecuteScalar();

    SqlChangeMonitor monitor = new SqlChangeMonitor(dep);

    policy.ChangeMonitors.Add(monitor);
   }
  }

  MemoryCache.Default.Add("MaintenanceMode", inMaintenanceMode, policy);
 }
 else
 {
  inMaintenanceMode = (bool)MemoryCache.Default.Get("MaintenanceMode");
 }

 return inMaintenanceMode;
}

This code is a simple way to cache a value that specifies whether the application is currently in maintenance mode. The dbo.Maintenance table contains a single row with a single bit column. This code will allow your application to continuously check to see if it should go into maintenance mode, without hammering your database.

When the value changes in the database, the application receives a notification that it should invalidate the cache. Then, in the next call to IsInMaintenanceMode, MemoryCache.Default["MaintenanceMode"] returns null, causing it to re-register the notification. Just what we want.

Notes

  • You MUST call SqlDependency.Start first, otherwise it just doesn't work.
  • Your SQL Command MUST follow the guidelines located at http://msdn.microsoft.com/en-us/library/ms181122(SQL.100).aspx. There are lots of things to consider about how you build your query, so pay close attention to this document.
  • After adding your command object to the SqlDependency object, you MUST execute the command at least once, otherwise it will not register the notification.
  • After executing the command once, you can dispose of your connection. Behind the scenes, .Net will keep a connection open to your SQL server to listen for the notification.

I hope this helps some people out with dealing with this. I know I spent WAY too much time looking for documentation that just didn't exist.

Sunday, November 22, 2009

Google Code Upload Utility

Summary

Attached is a utility written in C# that you can use to upload files to a code.google.com project.

Full Source Code

You can get the source code at my Google code project.

Usage

GCUpload.exe -u:<username> -p:<password> -n:<projectname> -s:<summary> -f:<filename> [-t:<targetfilename>] [-l:<label1>[,<label2>,...]] [--url:<uploadurl>] [--quiet-q]

Partial Source

The main part of this application is the class below:

/*
    GCUpload - Uploads a file to a googlecode.com project.
    Copyright (C) 2009 Brodrick E. Bassham, Jr.

    This program is free software: you can redistribute it and/or modify
    it under the terms of the GNU General Public License as published by
    the Free Software Foundation, either version 3 of the License, or
    (at your option) any later version.

    This program is distributed in the hope that it will be useful,
    but WITHOUT ANY WARRANTY; without even the implied warranty of
    MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
    GNU General Public License for more details.

    You should have received a copy of the GNU General Public License
    along with this program.  If not, see <http://www.gnu.org/licenses/>.

*/

using System;
using System.Collections.Generic;
using System.IO;
using System.Net;
using System.Text;
using System.Text.RegularExpressions;

namespace GCUpload
{
    public class FileUpload
    {
        private static readonly string BOUNDARY = "BASSHAMMAHSSAB";
        private static readonly string URL_FORMAT = "https://{0}.googlecode.com/files";
        private static readonly string METHOD = "POST";
        private static readonly string CONTENT_TYPE = string.Format("multipart/form-data; boundary={0}", BOUNDARY);
        private static readonly string USER_AGENT = "Bassham GoogleCode Uploader v1.0";

        public FileUpload()
        {
        }

        private static Uri GetUploadUrl(string url, string projectName)
        {
            if (!string.IsNullOrEmpty(url))
            {
                return new Uri(url);
            }
            else
            {
                if (string.IsNullOrEmpty(projectName))
                {
                    throw new ArgumentException("You must supply either a url or a projectName");
                }

                return new Uri(string.Format(URL_FORMAT, projectName));
            }
        }

        private static string GetAuthToken(string userName, string password)
        {
            return Convert.ToBase64String(Encoding.UTF8.GetBytes(string.Format("{0}:{1}", userName, password)));
        }

        private static void WriteToStream(Stream dest, string str)
        {
            byte[] buffer = Encoding.UTF8.GetBytes(str);

            dest.Write(buffer, 0, buffer.Length);
        }

        /// <summary>
        /// Uploads a file to a specified Google Code project.
        /// </summary>
        /// <param name="userName">Your googlecode.com username.</param>
        /// <param name="password">Your googlecode.com password.  NOTE: This is NOT your gmail or other google account password.  This is the password you use to connect to SVN.  It can be found at http://code.google.com/hosting/settings/ .</param>
        /// <param name="projectName">The googlecode.com Project.</param>
        /// <param name="fileName">The path to the file you want to upload.</param>
        /// <param name="summary">A description of the file.  This must not be null.</param>
        public static void UploadFile(
            string userName,
            string password,
            string projectName,
            string fileName,
            string summary)
        {
            UploadFile(userName, password, projectName, fileName, null, summary, null, null);
        }

        /// <summary>
        /// Uploads a file to a specified Google Code project.
        /// </summary>
        /// <param name="userName">Your googlecode.com username.</param>
        /// <param name="password">Your googlecode.com password.  NOTE: This is NOT your gmail or other google account password.  This is the password you use to connect to SVN.  It can be found at http://code.google.com/hosting/settings/ .</param>
        /// <param name="projectName">The googlecode.com Project.</param>
        /// <param name="fileName">The path to the file you want to upload.</param>
        /// <param name="targetFileName">The name you want the file to have once uploaded.</param>
        /// <param name="summary">A description of the file.  This must not be null.</param>
        /// <param name="labels">A list of labels to give the uploaded file.  Example: A label of "Featured" will make the download show up on the project home page.</param>
        /// <param name="uploadUrl">The URL to upload the file to.  Usually you will set this to null and it will be generated from the project name.  Do not set this unless you know what you are doing.</param>
        public static void UploadFile(
            string userName,
            string password,
            string projectName,
            string fileName,
            string targetFileName,
            string summary,
            List<string> labels,
            string uploadUrl)
        {
            if (string.IsNullOrEmpty(fileName))
            {
                throw new ArgumentException("You must specify a file to upload.", "fileName");
            }

            if (string.IsNullOrEmpty(summary))
            {
                throw new ArgumentException("You must specify a description of the file.", "summary");
            }

            if (string.IsNullOrEmpty(targetFileName))
            {
                targetFileName = Path.GetFileName(fileName);
            }

            Uri uploadUri = GetUploadUrl(uploadUrl, projectName);

            System.Diagnostics.Trace.WriteLine(uploadUri.ToString());

            HttpWebRequest req = WebRequest.Create(uploadUri) as HttpWebRequest;

            req.Method = METHOD;
            req.ContentType = CONTENT_TYPE;
            req.UserAgent = USER_AGENT;

            req.Headers.Add(HttpRequestHeader.Authorization, string.Format("Basic {0}", GetAuthToken(userName, password)));

            System.Diagnostics.Trace.WriteLine("Connecting...");

            using (Stream reqStream = req.GetRequestStream())
            {
                StringBuilder sb = new StringBuilder();

                sb.AppendLine(string.Format("--{0}", BOUNDARY));
                sb.AppendLine("Content-Disposition: form-data; name=\"summary\"");
                sb.AppendLine();
                sb.AppendLine(summary);

                if (labels != null && labels.Count > 0)
                {
                    foreach (string label in labels)
                    {
                        sb.AppendLine(string.Format("--{0}", BOUNDARY));
                        sb.AppendLine("Content-Disposition: form-data; name=\"label\"");
                        sb.AppendLine();
                        sb.AppendLine(label);
                    }
                }

                sb.AppendLine(string.Format("--{0}", BOUNDARY));
                sb.AppendLine(string.Format("Content-Disposition: form-data; name=\"filename\"; filename=\"{0}\"", fileName));
                sb.AppendLine("Content-Type: application/octet-stream");
                sb.AppendLine();

                WriteToStream(reqStream, sb.ToString());

                using (FileStream f = File.OpenRead(fileName))
                {
                    int bufferSize = 4096;
                    byte[] buffer = new byte[bufferSize];
                    int count = 0;

                    while ((count = f.Read(buffer, 0, bufferSize)) > 0)
                    {
                        reqStream.Write(buffer, 0, count);
                    }
                }

                sb = new StringBuilder();

                sb.AppendLine();
                sb.AppendLine(string.Format("--{0}--", BOUNDARY));

                WriteToStream(reqStream, sb.ToString());
            }

            try
            {
                using (HttpWebResponse res = req.GetResponse() as HttpWebResponse)
                {
                    System.Diagnostics.Trace.WriteLine(res.Headers.ToString());
                    System.Diagnostics.Trace.WriteLine(new StreamReader(res.GetResponseStream()).ReadToEnd());
                }
            }
            catch (WebException ex)
            {
                if (ex.Response != null)
                {
                    HttpWebResponse res = ex.Response as HttpWebResponse;

                    System.Diagnostics.Trace.WriteLine(res.StatusDescription);
                    string responseBody;

                    using (StreamReader rdr = new StreamReader(res.GetResponseStream()))
                    {
                        responseBody = rdr.ReadToEnd();

                        responseBody = Regex.Replace(
                            Regex.Replace(responseBody, "<br(\\s*/*)>", "\\n"),
                            "(<.*?>)", string.Empty).Replace(" ", " ");

                        System.Diagnostics.Trace.WriteLine(responseBody);
                    }

                    // Put the response body in the exception, since google puts the actual error there.
                    throw new Exception(responseBody, ex);
                }
            }
        }

        /// <summary>
        /// Uploads a file to a specified Google Code project.
        /// </summary>
        public void UploadFile()
        {
            FileUpload.UploadFile(this.UserName, this.Password, this.ProjectName, this.FileName, this.TargetFileName, this.Summary, this.Labels, this.UploadUrl);
        }

        /// <summary>
        /// Your googlecode.com username.
        /// </summary>
        public string UserName { get; set; }

        /// <summary>
        /// Your googlecode.com password.  NOTE: This is NOT your gmail or other google account password.  This is the password you use to connect to SVN.  It can be found at http://code.google.com/hosting/settings/.
        /// </summary>
        public string Password { get; set; }

        /// <summary>
        /// The googlecode.com Project.
        /// </summary>
        public string ProjectName { get; set; }

        /// <summary>
        /// The path to the file you want to upload.
        /// </summary>
        public string FileName { get; set; }

        /// <summary>
        /// The name you want the file to have once uploaded.
        /// </summary>
        public string TargetFileName { get; set; }

        /// <summary>
        /// A description of the file.  This must not be null.
        /// </summary>
        public string Summary { get; set; }

        /// <summary>
        /// A list of labels to give the uploaded file.  Example: A label of "Featured" will make the download show up on the project home page.
        /// </summary>
        public List<string> Labels { get; set; }

        /// <summary>
        /// The URL to upload the file to.  Usually you will set this to null and it will be generated from the project name.  Do not set this unless you know what you are doing.
        /// </summary>
        public string UploadUrl { get; set; }
    }
}

Sunday, March 1, 2009

SharpSvn: .Net Bindings for Subversion

Are you a .Net developer using subversion for your source control system? Have you ever wanted to create your own tools to interface with subversion using .Net code? SharpSvn is a binding of the Subversion Client API for .Net 2.0 applications contained within a set of xcopy-deployable dll's. These allow you to write .Net code to interface with your subversion repository. Use it to create an application that periodically checks the status of your repository and emails you a diff of the recent check-ins. Or use it to parse your log messages to find a certain commit.

Coming soon I will show you how to use SharpSvn to create a simple application that interfaces with a subversion repository.

Saturday, February 28, 2009

TransformXml - A quick app to transform an xml file with an xsl file.

This is a real quick and dirty way to transform an XML file using an XSL file.

using System;
using System.Xml;
using System.Xml.Xsl;

namespace TransformXml
{
 class Program
 {
  static void Main(string[] args)
  {
   if (args.Length != 3)
   {
    Console.WriteLine("TransformXml Usage:");
    Console.WriteLine("  TransformXml.exe <source xml file> <xsl file> <output file>");
    return;
   }

   string sourceXmlFile = args[0];
   string xslFile = args[1];
   string outputFile = args[2];

   XslCompiledTransform t = new XslCompiledTransform();

   t.Load(xslFile);

   t.Transform(sourceXmlFile, outputFile);
  }
 }
}

RPT to CSV

A simple console application to convert the fixed-width output from the "Output to File" option in SQL Server Management Studio to a CSV file that is actually useful.

I use this instead of the CSV setting in Management Studio because I like seeing the "Output to Text" option in fixed-width format.

This application will not work if the output has newlines or carriage returns in it.

namespace RptToCsv
{
    using System;
    using System.IO;

    /// <summary>
    /// The main class for the RptToCsv program.
    /// </summary>
    public class Program
    {
        /// <summary>
        /// The main entry point to the application.
        /// </summary>
        /// <param name="args">Command line arguments.</param>
        internal static void Main(string[] args)
        {
            if (args.Length > 0)
            {
                for (int i = 0; i < args.Length; i++)
                {
                    string inputFile;
                    string outputFile;

                    inputFile = args[i];
                    outputFile = Path.GetFileNameWithoutExtension(args[i]) + ".csv";

                    Environment.CurrentDirectory = Path.GetDirectoryName(inputFile).Length == 0 ? Environment.CurrentDirectory : Path.GetFullPath(Path.GetDirectoryName(inputFile));

                    using (StreamReader inputReader = File.OpenText(inputFile))
                    {
                        string firstLine = inputReader.ReadLine();
                        string secondLine = inputReader.ReadLine();

                        string[] underscores = secondLine.Split(new char[] { ' ' }, StringSplitOptions.RemoveEmptyEntries);

                        string[] fields = new string[underscores.Length];
                        int[] fieldLengths = new int[underscores.Length];

                        for (int j = 0; j < fieldLengths.Length; j++)
                        {
                            fieldLengths[j] = underscores[j].Length;
                        }

                        int fileNumber = 0;

                        StreamWriter outputWriter = null;

                        try
                        {
                            outputWriter = File.CreateText(outputFile.Insert(outputFile.LastIndexOf("."), "_" + fileNumber.ToString()));
                            fileNumber++;

                            int lineNumber = 0;

                            WriteLineToCsv(outputWriter, fieldLengths, firstLine);
                            lineNumber++;

                            string line;

                            while ((line = inputReader.ReadLine()) != null)
                            {
                                if (lineNumber >= 65536)
                                {
                                    outputWriter.Close();
                                    outputWriter = File.CreateText(outputFile.Insert(outputFile.LastIndexOf("."), "_" + fileNumber.ToString()));
                                    fileNumber++;

                                    lineNumber = 0;

                                    WriteLineToCsv(outputWriter, fieldLengths, firstLine);
                                    lineNumber++;
                                }

                                if (!WriteLineToCsv(outputWriter, fieldLengths, line))
                                {
                                    break;
                                }

                                lineNumber++;
                            }
                        }
                        catch (Exception ex)
                        {
                            Console.WriteLine(ex);
                            Console.WriteLine("NOTE: Input file must not have any newline characters as field contents.");
                            Console.WriteLine();
                            Console.WriteLine("Press any key to continue...");
                            Console.ReadKey(true);
                        }
                        finally
                        {
                            if (outputWriter != null)
                            {
                                outputWriter.Close();
                            }
                        }

                        // If we only had one file created, we don't need the file number in the name.
                        if (fileNumber == 1)
                        {
                            try
                            {
                                if (File.Exists(outputFile))
                                {
                                    File.Delete(outputFile);
                                }

                                File.Move(outputFile.Insert(outputFile.LastIndexOf("."), "_0"), outputFile);
                            }
                            catch (Exception ex)
                            {
                                Console.WriteLine(ex);
                                Console.WriteLine("Press any key to continue...");
                                Console.ReadKey(true);
                            }
                        }
                    }
                }
            }
            else
            {
                Console.WriteLine("Converts the ouput of a SQL Server Management Studio .rpt file to a CSV file.");
                Console.WriteLine("You can generate a .rpt file by selecting \"Results to File\" in the toolbar.");
                Console.WriteLine();
                Console.WriteLine("Usage: RptToCsv.exe <inputFile1> [<inputFile2> ...]");
                return;
            }
        }

        /// <summary>
        /// Converts a single line of fixed width fields to a single line of comma separated fields.
        /// </summary>
        /// <param name="outputWriter">The stream to write to.</param>
        /// <param name="fieldLengths">An array containing the lengths of the fixed with fields.</param>
        /// <param name="line">The line of fixed width fields to be converted to CSV.</param>
        /// <returns>True if it successfully converts the line, otherwise False.</returns>
        private static bool WriteLineToCsv(StreamWriter outputWriter, int[] fieldLengths, string line)
        {
            if (line.Length == 0)
            {
                return false;
            }

            int index = 0;

            for (int i = 0; i < fieldLengths.Length; i++)
            {
                string value;

                if (i < fieldLengths.Length - 1)
                {
                    value = line.Substring(index, fieldLengths[i]);
                }
                else
                {
                    value = line.Substring(index);
                }

                value = value.Replace("\"", "\"\"");
                value = value.Trim();

                if (value == "NULL")
                {
                    value = string.Empty;
                }

                outputWriter.Write("\"{0}\"", value);
                index += fieldLengths[i] + 1;

                if (i < fieldLengths.Length - 1)
                {
                    outputWriter.Write(",");
                }
                else
                {
                    outputWriter.WriteLine();
                }
            }

            return true;
        }
    }
}

xLibrary - Using a custom IHttpHandler to access embedded javascript

I have written an article at CodeProject.com showing in depth how to embed your JavaScript files in your ASP.Net assemblies and how to implement IHttpHandler to get them out.