Database

Using XML to perform database operations rather than parameterized query?

Bigyan Rajbhandari — Sun, 24 Aug 2008 18:55:16 GMT

There's no doubt that today xml is becoming more and more prevalent as platform-independent data format. Starting SQL server 2005, Microsoft also added tons of support for xml and a new native data type XML, since then I have started to see lots of applications that utilize this data type to run very complex solutions. A new trend is starting to emerge, as one of my colleagues calls it a "paradigm shift", where applications use xml serialized objects (that map to tables) to perform inserts/updates to table rather than having to pass each parameter independently. However, the question remains if this approach is necessarily better than previous?

For example consider the following scenario:
In middle-Tier (.Net Code) my object looks like the following

My serialized xml looks like below:
<Customer customerID="12" customerName="Bigyan" address="123 Redmond Way" phone="123-123-3434"/>

And my call to database stored proc like:
exec Customer_UpdateInformation @customerXml

In parameterized query I would have done something like this:
exec Customer_UpdateInformation @customerID, @customerName, @addresss, @phone

There are both pros and cons to this approach, which I'll explain next. But I also want to clarify that I am NOT advocating one approach vs other, but actually want your opinion on this approach.

Pros:

Instead of passing 4 parameters, I need to pass only 1. Thus my signature for the database contract is much more succinct.
If I need to add or modify field, I have less code to deal with and do NOT need to change my signature
If I am just performing updates on certain fields (not all), I might be able to pass only updated fields and reduce my message size
Serialization and Deserialization on my .Net objects can be automated, thus reducing effort to map those fields with database fields.
If I wanted to perform bulk updates, then I have ability to pass multiple elements in my xml; this greatly reduces chatting between .Net and Sql Server.

Cons:

First and foremost, I'll lose type safety on the parameters passed.
Xml parsing is little expensive (not a whole lot), compared to primitive data types. Bulk process might make up ground in this area.
Data contract I am exposing through the stored procedure is not self explanatory. Just looking at the parameter, no one will know what to pass.

Is V1 of LINQ to SQL only suitable for RAD?

Bigyan Rajbhandari — Tue, 25 Mar 2008 03:31:38 GMT

In my previous couple posts, I have been religiously trying to solve the problem or should I say 'find a best pattern' to use with linq to sql. In past few projects, I ventured into Linq to sql world abandoning ORM tools (OR mapper, NHibernate…) that I had been using; however, I have been greatly disappointed by the fact that Linq to sql doesn't support few fundamental concepts/patterns (LINQ to SQL takes a little different approach than traditional ORM tools). I am starting to have serious doubts if V1 of Linq2SQl is even cut out for any n-tier application (I am not even thinking enterprise application). Don't get me wrong, LINQ is a great framework and LINQ to SQL is an awesome extension. But in the real world business application the implementation of LINQ to SQL out of the box is not practical. How did I arrive to this conclusion? Here's how:

The two fundamental problems it fails to support out of the box are:

1. Working with objects in LINQ to SQL in disconnected context is painful. i.e. Most mid-large size applications are designed as n-tier architecture, thus entities within LINQ act more as a container (DTO or Business objects) for carrying data from one tier to another. The problem with LINQ to SQL is that disconnecting from one datacontext and reattaching to another is not trivial. This is a serious drawback given the fact that almost all the applications needs to use this model other than "quick demos".

I have seen lots of posts/articles from developers questioning the same problem. Various people have taken a stab at solving this, if you are seriously considering LINQ to SQL read these.

Benjamin's ULinqGen project in codeplex allows change tracking in pure POCO models
Post by Rick Strahl, explaining problem in Linq to SQL in a disconnected environment.
Matt's Linq2SqlEB project in codeplex which implements synchronization.

2. If you somehow implement an architecture to have live datacontext, one serious issue that you need to focus on is the lifetime management of DataContext. Please be aware, that poorly managed datacontext will bloat eventually and will hamper performance.

See my previous post on managing lifetime of datacontext
Rick Strahl discusses datacontext lifetime management here
Dino Esposito, suggest that you should use atomic context, but I don't see how it would solve the problem if you can't work in a disconnected environment.

Conclusion:
Am I ready to make jump to LinqToSql in my next project from my previous ORM tools? I don't know about others but, I will be holding back until I figure out a best way to work in a disconnected environment, which might be the next project. But the point is disconnected context support will be required. Dinesh Kulkarni, a senior dev for Linq to SQL team advises not use Active Record Pattern, but the problem is I don't see any pattern that I can use effectively in a n-tier architecture, not only the Active Record. LINQ to SQL itself uses great patterns within, but the way datacontext is managed and Entities/child entities are persisted it presents a different problem. It makes for a great presentation or RAD applications so far, but fails to support n-tier applications easily. I am considering concepts of offline datacontext and will follow up with more posts on this.

Is ADO.Net Entity Framework an answer?
Not really. The framework uses same concept of active context as Linq to SQL and quickly runs into the problem in a disconnected environment. I haven't done an in-depth implementation of it but the problem seems to be the same.

Let me know what you guys think of LINQ to SQL.

(Follow Up) Best practice and effective use of DataContext in LINQ

Bigyan Rajbhandari — Thu, 20 Mar 2008 21:46:24 GMT

NOTE: 3/24/2008 - The problem of working with LINQ to sql extends beyond lifetime management of DataContext, thus I am scrapping the second part of my blog. Only thing useful in this post is the performance comparision. Here's new post on LINQ to SQL

In my previous post, I had probed a question on how to effectively incorporate DataContext in ones pattern so the ease of use and unit of work is well represented. Thanks to everyone who provided feedback, it definitely was a good read and provided lots of insight. Few reasons we got into asking this question instead of using simple create/dispose model, in the first place was

We wanted to make it so that it was easy to use
We heavily use Repository Pattern for business logic/transaction scoping and Entity model for CRUD operations, so wanted to make sure datacontext we used was as efficient as possible.
Concerns regarding efficiency of creating/disposing datacontext and not being able to use caching of datacontext effectively.

~~However, after reading several posts and looking deeper into datacontext, it seems that creating/disposing it as required is the best way to go~~. In order, to get rid of any suspicion I even created a little test to compare Ambient Context and Instantiation as required model, results are below. In the image below, first method GetByID(int32) actually instantiates datacontext everytime, where as the second one takes in a datacontext instantiated once by the caller (code is included at the bottom of the post). The results are impressive for dynamic instantiation. 23.7 secs vs. 20.9 secs for 6200 iterations. I think 3 secs is a reasonable margin not taking into account impact in memory with ambient datacontext or the efficiency of caching.

Either way, I believe for our purpose and in general a repository pattern incorporated with entity model should work pretty well. One thing Jeff did point out to me however was, once you have disposed your datacontext, any attempt to load/access child objects that is marked for lazy load will throw disposed object error as below. So make sure you load your objects while the context is alive, once that is done subsequent access is successful.

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Timers;
using System.Diagnostics;

namespace ConsoleApplication4
{
    class Program
    {
        static void Main(string[] args)
        {
//Create & dispose Context everytime
            Test1();
            
        //Ambient Context Concept
Test2();
        }

//Create datacontext everytime
        private static void Test1()
        {
            List<EcardOrder> orders;
            using (MercyDataContext dbContext = new MercyDataContext())
            {
                for (int i = 0; i < 100; i++)
                {
                    orders = dbContext.EcardOrders.ToList();
                    foreach (EcardOrder order in orders)
                    {
                        //Datacontext will be created everytime 
// in this function
                        Ecard card = Ecard.GetByID(order.EcardID);
                    }
                }
            }
        }

        
//Create datacontext only once
        private static void Test2()
        {
            List<EcardOrder> orders;
            using (MercyDataContext dbContext = new MercyDataContext())
            {
                for (int i = 0; i < 100; i++)
                {
                    orders = dbContext.EcardOrders.ToList();
                    foreach (EcardOrder order in orders)
                    {
                        //Pass datacontext to the function as well
                        Ecard card = Ecard.GetByID(order.EcardID, dbContext);
                    }
                }
            }
        }


    }

    /// <summary>
    /// Partial Ecard class extended from database
    /// </summary>
    public partial class Ecard
    {
        //new datacontext is created and disposed everytime.
        public static Ecard GetByID(int ID)
        {
            using (MercyDataContext _context = new MercyDataContext())
            {
                return _context.Ecards.SingleOrDefault(e => e.EcardID == ID);
            }
        }

        //Puesdo concept of ambient datacontext
        //NOTE: once the Ecard is loaded it is cached in the context, 
        //thus items will be returned from cache
        public static Ecard GetByID(int ID, MercyDataContext _context)
        {
            return _context.Ecards.SingleOrDefault(e => e.EcardID == ID);
        }
    }


}

Best practice and effective way of using DataContext in Linq to SQL?

Bigyan Rajbhandari — Wed, 19 Mar 2008 01:51:33 GMT

NOTE: See my follow up post here for more info.

At work, Jeff and I have been throwing around ideas to find a best way to implement DataContext in Linq so that we can integrate it into the base class in our framework while achieving following goals.

Implementation should be easy and non-redundant, so that we do not need to do new DataContext(), every time we have to use one.
Portability of the DataContext should be such that we do not need to pass it around as parameters from one tier to another.
"Unit of work" pattern should be well represented and "very" transparent.
Context should not be persisted without intent, or be open to manipulation from other methods, such that "unit of work" is compromised or a transaction is ill-represented

These seem like simple goals given the features and flexibility of LINQ, however, in reality this has become much more difficult and annoying to achieve than we previously thought. Here are several ideas we have been kicking around, and problem it presents. I would like to hear from people who have effectively used LINQ Datacontext in these scenarios or in a pattern that's most effective.

1. Creating and disposing LINQ as required.

DbDataContext myContext = new DbDataContext();

//your code goes here

myContext.SubmitChanges();

Problems with this pattern:

Creating datacontext everytime we need to use is cumbersome and in my opinion just verbose.
If you have to use datacontext in any of the method this code calls, you'll have to pass the datacontext with it. This is one- too many parameters to pass around

2. Creating a static DataContext and using it throughout the application.

DataLayer.DataContext.Current

Problems with this pattern:

Creating static object isn't always a good thing. In this case, the context is available for manipulation at any level. And unit of work quickly becomes very hard to maintain
If manipulated by other threads or methods context isn't clean anymore and cannot be trusted

3. Ambient DataContext (Currently using this)

The idea behind ambient datacontext is that context is created for a particular thread or a httpcontext(in asp.net) as soon as there is a first call for the DataContext. Then the same context is used for that thread or request unless it is disposed manually. This is done by storing the context in the request/thread data store. The approach to this pattern is similar to static DataContext, however, separation is provided for each thread and requests. This works really well in Asp.Net, however, is again plagued by some of the problems of static context.

Problems with this pattern:

The context is available for manipulation at any level. And quickly becomes very hard to maintain unit of work
Portability across thread is very hard
Unit of work pattern is not transparent enough

4. Finally, the last idea we've been trying to come up with is concept similar to TransactionScope.

In this pattern, you would used using{ } block to initialize your context and represent a "Datacontext Scope". Any method below that will automatically be able to figure out the proper context, through thread/request datastore. Then just before the dispose happens, you manually call the SubmitChanges() method. This is the one I like so far, however, has few practical problems.

Problems with this pattern:

If sub method needs to implement a new context, we need to figure out how to exclude it from the parent context's using block. If allowing multiple contexts in multiple using blocks, how do you determine which context to use in sub-sub methods?

Please do comment on what you think is the best approach to solving this problem.