Data Sources

This section describes the reasons for and against using three of the most common data source types, along with an overview of the other types available.

Databases

When you start thinking about data sources, the most obvious one that springs to mind is the database, which will generally provide the most reliable, scaleable, and secure option for data storage. When you're dealing with large amounts of data, databases also offer the best performance. However, the very fact that other solutions exist is a sure indication that in some circumstances, they're not the best choice.

In general, databases are designed to store large amounts of data in a manner that allows arbitrary quantities of data to be retrieved in arbitrary order. For small collections of data, such as a set of contact details, the time and other costs involved in creating and accessing a database might outweigh the benefits that databases provide.

A company employee in a database might move us to create a table called Employee that can contain the same pieces of data about a number of employees. Such information could include their EmployeeID (number), LastName, FirstName, BirthDate, and Country:
 

Text Files

At the opposite end of the scale from using databases to store information for a web site is the use of text files. Although text files can store information in almost any conceivable format, they are generally used for storing a set of data, one item on each line. If we were to capture the employee information detailed above, we could store the LastName, FirstName, BirthDate, and Country of two employees in a text file as follows:

   Smith, John, 05-04-1979, UK
   Bloggs, Joe, 29-09-1981, US

For simple information such as this, a text file provides an easy way of reading and writing data. If the data to be stored has more structure, however, it becomes far more time consuming. For example, it could be the case that each of these employees has placed an order for some office supplies. Rather than adding all of that information to the text file as well, it would be better to hold it separately, and then define relationships between the two sets of data.

When the data starts to gain 'structure' in this manner, a method of giving the file itself some structure must be found, and a way of retrieving it and representing it in memory must also implemented. One way of doing this is through the use of XML.

XML

In some ways, XML documents can be thought of as a stepping-stone between text files and databases; they store data using text files, but use a hierarchical and relational format that is both extensible and self-describing, providing a number of the benefits of a database system. Before we go any further in explaining the use of XML as a data source, a sample fragment of an XML document is shown below:

   
     
       
                 BirthDate="05-04-1979" Country="UK" />
       
                 BirthDate ="29-09-1981" Country="US" />
     
   

As you can see, the same information is being stored as in the text file, but there's also an indication of the nature of that information. You know that 29-09-1981 is the BirthDate of Joe Bloggs, because the data says so. Another benefit of XML is that it can contain multiple types of information in one document; a fragment like the one below could be inserted after :

   
     
       Staples
       Pencils
     
     
       Biros
       Erasers
     
   

Using the comprehensive functionality that's built into the XML-handling support provided by the .NET Framework (and other platforms), retrieving and manipulating the orders separately from the employees can be accomplished quite easily. This makes it possible to specify an order from the list for each employee by storing the ID of each order as part of the employee's details:

   
             BirthDate="05-04-79" Country="UK" Order="2" />

XML is a powerful way of representing information, but in some circumstances performance can be a problem: updating and retrieving data from XML can be a time-consuming process. This is rarely an issue when a few users are accessing a small amount of data, but if there's a lot of data (or a lot of users) it can sometimes become one.

Other Sources

Between them, the three options enumerated above cover the main categories of data store, but there are many others that either fall between these, or follow a completely different paradigm. Most of the types that we haven't covered, though, are domain-specific - that is, that they've been developed to suit a specific task. On the Windows platform, typical examples of these include:

• Microsoft Exchange Server - the data store containing e-mail, calendar, and contact information
• Active Directory - the information that's stored by Windows-based servers regarding the users on the system, their permissions, etc.
• Spreadsheets - applications such as Excel store their data in a tabular format, a grid containing values that are used in tasks such as financial calculations.

In summary, although this book is focusing on databases (and uses them in the majority of its examples), it is important to remember that databases are not the only kind of data store, and that other mechanisms for storing data can often achieve the same goal more efficiently.