Innovative Games

One of the big features any useful engine needs is the ability to save its state and the state of its components to file. This is necessary for things like saving a game, but is more useful for things like saving a scene in an editor. C# provides two types of serializer.

The first is a binary serializer. This is useful, but it will save more information than we want, and in a way that isn’t very useful to a person reading it. (This is better for security, but there are ways to encrypt more legible files later on if necessary).

The second is an Xml serializer. This is more helpful to us, but it needs to know what type of object to serialize before it can serialize. Thus, we can’t use this because we don’t know what type of objects the scene will contain beforehand.

The final option is to create our own serializer. This is probably the best solution here because we can have it save whatever we want in any format, and we can save multiple objects in a file, along with data about the state of the Engine.

Our serialize will work like this:

class: SerializationData – This class is effectively a glorified Dictionary, except that it has some more code in it to keep track of Assemblies used by the objects it contains.

- method: void AddData(string Key, object Data) – Adds data to the SerializationData
- method: object GetData(string Key) – Gets data back from a SerializationData built by the Serializer
- method: void AddDependency(Type type) – Tells the Serializer make sure that the Assembly used by the Type will be available when deserializing
- method: Type GetTypeFromDependency(string Type) – Gets the Type back from the Serializer, which will get it from the corresponding Assembly

class: Serializer – This is the class that handles the serialization of objects to file. It saves everything in an XML format.

The process of serialization works like this:

- An XmlWriter is created
- A Serializer is created
- The Serializer writes information about the current GameScreens to file
- For each component in the scene
- A SerializationData object is retrieved from the Component
- The Serializer saves the data in the SerializationDtaa object to file
- When writing data, the serializer keeps track of which Assemblies the Types in the data are from, so they can be loaded when deserializing
- The list of Types and Assemblies used are written to file

The process of deserialization works pretty much the opposite direction

- The list of dependent Types and Assemblies are re-loaded
- The GameScreens are recreated
- All the components are recreated in the correct GameScreen and given back the SerializationData they provided about themselves

This design allows us to save (almost) any type of component, and only the information about it that we want to save. It does mean we will have to add code to every component to save and reload its state, but this is actually a good thing because we will have more control over the process of creating components from a file.

A few important notes:

- The following implementation cannot save a List, Dictionary, etc. It could be extended to do so, but we will not cover this. I would recommend just pre-expanding lists by creating a long string with commas, or a similar method.

- The data saved should not be too complex, and any properties of the object saved should be public members, not properties (ie: not get/set accessor type properties, just public objects). An attempt will be made at saving more complex objects, but there are no gaurantees here. Types like Vector3, BoundingBox, etc. are okay, but don’t try to save a Camera object. The Camera should instead save data like its position, rotation, etc. as less complex Types. This not only makes it easier on the Serializer, but keeps the file size down.

- Data to be serialized will eventually be broken down into the basic types: int, float, byte, string, etc.

Let’s get started.

Before we can get started on the serialization, we need a way to reset the engine, for loading new scenes, etc. Add this function to the Engine class:

// Resets the Engine to its initial state
public static void Reset()
{
    List destroy = new List();

    foreach (GameScreen screen in Engine.GameScreens)
        foreach (Component component in screen.Components)
            destroy.Add(component);

    foreach (Component component in destroy)
        component.DisableComponent();

    List screenDestroy = new List();

    foreach (GameScreen screen in Engine.GameScreens)
        if (screen != Engine.BackgroundScreen)
            screenDestroy.Add(screen);

    foreach (GameScreen screen in screenDestroy)
        screen.Disable();

    Engine.Services.Clear();
    Engine.Content.Unload();
}

Now we can create some classes that assist in serialization- SerializationData and ServiceData:

// Information about Services
struct ServiceData
{
    public string Type;
    public bool IsService;
}

// Provides a link between the Component class and Serializer class. A component
// adds data and keys to this class to simplify serialization
public class SerializationData
{
    // Dictionary that stores the data. It is public so the Serializer can save
    // the data it holds.
    public Dictionary Data = new Dictionary();

    // Add data to the dictionary with the string key Key and value object Data
    public void AddData(string Key, object Data)
    {
        this.Data.Add(Key, Data);
    }

    // Get data of type T from the dictionary with the string key Key
    public T GetData(string Key)
    {
        object val = this.Data[Key];
        return (T)val;
    }

    // Whether or not the data contains the specified Key
    public bool ContainsData(string Key)
    {
        return Data.ContainsKey(Key);
    }

    public XmlWriter Writer
    {
        get { return Writer; }
    }

    // The serializer being used for serialization/deserialization
    Serializer serializer;
    XmlWriter writer;

    // Constructor sets the serializer being used
    public SerializationData(Serializer Serializer, XmlWriter Writer)
    {
        serializer = Serializer;
        writer = Writer;
    }

    // Tell the serializer we will be using the specified Type
    public void AddDependency(Type type)
    {
        serializer.Dependency(type);
    }

    // Get the Type specified through the serializer, so the right
    // Assembly will be used.
    public Type GetTypeFromDependency(string type)
    {
        return Assembly.Load(serializer.DependencyMap[type]).GetType(type);
    }
}

Now we can start on Serialize:

// Class used to serialize Components to an Xml file
public class Serializer
{
}

Next we’ll add some code to work with dependencies. In this case a dependency is a Type, and the Assembly it came from.

// Keeps track of what Assembly Types are located in. The Type is the Key and the
// Assembly is the value
public Dictionary DependencyMap = new Dictionary();

// Uses the DependencyMap to write the dependencies to the Xml File
public void WriteDependencies(XmlWriter Writer)
{
    Writer.WriteStartElement("Dependencies");

    // Temporary list is used to keep track of the dependencies we need to write
    List assemblies = new List();

    // For each Type, add its Assembly to the assembly list if it has not been added
    foreach (string assembly in DependencyMap.Values)
        if (!assemblies.Contains(assembly))
            assemblies.Add(assembly);

    // Write each assembly to the file
    foreach(string assembly in assemblies)
    {
        Writer.WriteStartElement("Assembly");
        Writer.WriteAttributeString("Name", assembly);

        // Write each type in the Assembly as a child node
        foreach(string type in DependencyMap.Keys)
        {
            if (DependencyMap[type] == assembly)
            {
                Writer.WriteStartElement("Type");
                Writer.WriteAttributeString("Name", type);
                Writer.WriteEndElement();
            }
        }

        Writer.WriteEndElement();
    }
}

// Add the type to the DependencyMap
public void Dependency(Type type)
{
    string name = type.FullName;
    string assembly = type.Assembly.FullName;

    if (!DependencyMap.ContainsKey(name))
        DependencyMap.Add(name, assembly);
}

// Load back the Assemblies used from file
public void PopulateAssemblies(XmlNode DependenciesRoot)
{
    foreach (XmlNode Node in DependenciesRoot.ChildNodes)
        foreach (XmlNode child in Node.ChildNodes)
            // For each node, add the type to the list. Attribute 0 = type name,
            // attribute 1 = assembly name
            DependencyMap.Add(child.Attributes[0].Value, Node.Attributes[0].Value);
}

// Clear the list of dependencies
public void ClearDependencies()
{
    DependencyMap.Clear();
}

Next is the code that saves information on GameScreens:

// Write the GameScreens in Engine to file
public void WriteGameScreens(XmlWriter Writer)
{
    Writer.WriteStartElement("GameScreens");

    foreach (GameScreen screen in Engine.GameScreens)
    {
        // The background screen is created automatically, so we dont need to serialize it
        if (screen != Engine.BackgroundScreen)
        {
            Writer.WriteStartElement("GameScreen");
            Writer.WriteAttributeString("Name", screen.Name);
            Writer.WriteAttributeString("Type", screen.GetType().FullName);
            Dependency(screen.GetType());

            if (screen.BlocksInput) { Writer.WriteElementString("BlocksInput", null); }
            if (screen.OverrideInputBlocked) { Writer.WriteElementString("OverrideInputBlocked", null); }
            if (screen.BlocksUpdate) { Writer.WriteElementString("BlocksUpdate", null); }
            if (screen.OverrideUpdateBlocked) { Writer.WriteElementString("OverrideUpdateBlocked", null); }
            if (screen.BlocksDraw) { Writer.WriteElementString("BlocksDraw", null); }
            if (screen.OverrideDrawBlocked) { Writer.WriteElementString("OverrideDrawBlocked", null); }

            if (screen == Engine.DefaultScreen) { Writer.WriteElementString("DefaultScreen", null); }

            Writer.WriteEndElement();
        }
    }

    Writer.WriteEndElement();
}

Now we will add the function that serializes an object. This will be used for each component when serializing.

// Saves the data contained in an object
public void SerializeObject(XmlWriter Writer, object Input, string InstanceName)
{
    // We can't serialize a null object
    if (Input == null)
        return;

    // If we are dealing with a seperate field, we can save its name
    if (InstanceName != null)
    {
        Writer.WriteStartElement("Field");
        Writer.WriteAttributeString("Name", InstanceName);
    }

    Type t = Input.GetType();

    // Add the type's assembly to the dependencies if neccessary
    Dependency(t);

    // If we have a value type, we can save it directly
    if (t == typeof(short) ||
        t == typeof(long) ||
        t == typeof(float) ||
        t == typeof(decimal) ||
        t == typeof(double) ||
        t == typeof(ulong) ||
        t == typeof(uint) ||
        t == typeof(ushort) ||
        t == typeof(sbyte) ||
        t == typeof(int) ||
        t == typeof(byte) ||
        t == typeof(char) ||
        t == typeof(string) ||
        t == typeof(bool))
    {
        // Write the type of value, then the value of the value
        Writer.WriteAttributeString("Type", t.FullName);
        Writer.WriteValue(Input.ToString());
    }
    else
    {
        // If its not a value type, we need to break it dowm
        Writer.WriteStartElement(Input.GetType().FullName);

        // Serialize all fields recursively. This will break them
        // down automatically if they are not value types too.
        foreach (FieldInfo info in Input.GetType().GetFields())
            SerializeObject(Writer, info.GetValue(Input), info.Name);

        Writer.WriteEndElement();
    }

    // Write the end if this is not a one line value
    if (InstanceName != null)
        Writer.WriteEndElement();
}

The rest of the code is used for rebuilding objects from XML.

// Deserialize the component defined in the ComponentNode
public Component Deserialize(XmlNode ComponentNode)
{
    // Find the type from the component nodes name
    Assembly a = Assembly.Load(DependencyMap[ComponentNode.LocalName]);
    Type t = a.GetType(ComponentNode.LocalName);

    // Create an instance of the type and a SerializationData object
    Component component = (Component)Activator.CreateInstance(t);
    SerializationData data = new SerializationData(this, null);

    // For each field defined, get its value and add the field to the
    // SerializationData
    foreach (XmlNode child in ComponentNode.ChildNodes)
    {
        // Make name and object values, and get the name from the 0
        // attribute, "Name"
        string name = child.Attributes[0].Value;
        object value = null;

        // If the field node contains text only, it is a value type
        // and we can set object directly
        if (child.ChildNodes[0].NodeType == XmlNodeType.Text)
            value = parse(child);
        // Otherwise we need to recreate a more complex object from the data
        else if (child.ChildNodes[0].NodeType == XmlNodeType.Element)
            value = parseTree(child.FirstChild);

        // Save the field to the SerializationData
        data.AddData(name, value);
    }

    // Tell the component to load from the data
    component.RecieveSerializationData(data);

    return component;
}

// Returns an object from an XmlNode that contains a value type
object parse(XmlNode value)
{
    // Get the type being parsed
    Assembly a = Assembly.Load(DependencyMap[value.Attributes["Type"].InnerText]);
    Type t = a.GetType(value.Attributes["Type"].InnerText);

    // If it is a string, we can return it how it is
    if (t == typeof(string))
        return value.InnerText;

    // Otherwise, it can be parsed using the "Parse()" method all value
    // types have, invoked using reflection
    MethodInfo m = t.GetMethod("Parse", new Type[] { typeof(string) });

    // Return the value "Parse()" returns, using the node text
    // as the argument
    return m.Invoke(null, new object[] { value.InnerText });
}

// Returns an object constructed from a tree of XmlNodes
object parseTree(XmlNode root)
{
    // Get the type to be built
    Assembly a = Assembly.Load(DependencyMap[root.Name]);
    Type t = a.GetType(root.Name);

    // Create an instance of the type
    object instance = Activator.CreateInstance(t);

    // For each field in the node's children
    foreach (XmlNode member in root.ChildNodes)
    {
        // Get the info on it
        FieldInfo fInfo = t.GetField(member.Attributes["Name"].Value);

        // If the node contains a value type, set the value directly
        if (member.ChildNodes[0].NodeType == XmlNodeType.Text)
            fInfo.SetValue(instance, parse(member));
        // Otherwise, we need to parse it again as a tree. This will
        // do the same recursively if the parsed type isn't a value type
        else
            fInfo.SetValue(instance, parseTree(member));
    }

    return instance;
}

Now we’ll add the function to serialize a SerializationData object:

// Serialize the SerializationData to file
public void Serialize(XmlWriter Writer, SerializationData Input)
{
    foreach (KeyValuePair<string, object> pair in Input.Data)
    {
        Writer.WriteStartElement("Field");
        Writer.WriteAttributeString("Name", pair.Key);

        // We won't have an instance name here because
        // it is already in the Name attribute
        SerializeObject(Writer, pair.Value, null);

        Writer.WriteEndElement();
    }
}

Now that we will be saving components to file, we need a way to identify them. We can do this by adding a simple string for the component’s name. This name will automatically be uniquely set for each component, but components can be renamed later on if wanted.

static int count = 0;
public string Name;

public override string ToString()
{
    return this.Name;
}

// InitializeComponent()

count++;
Name = this.GetType().FullName + count;

Next Component needs to have some base functions to get serialization data. It will also have some virtual functions that derived components can override to add their own data.

// Returns a SerializationData a Serializer can use to save the state
// of the object to an Xml file
public SerializationData GetSerializationData(Serializer Serializer, XmlWriter Writer)
{
    // Create a new SerializationData
    SerializationData data = new SerializationData(Serializer, Writer);

    // Add the basic Component values
    data.AddData("Component.DrawOrder", DrawOrder);
    data.AddData("Component.ParentScreen", Parent.Name);
    data.AddData("Component.Visible", Visible);
    data.AddData("Component.Name", this.Name);

    // Tell the serializer that it will need to know the type of
    // component
    data.AddDependency(this.GetType());

    // Construct a ServiceData
    ServiceData sd = new ServiceData();

    // If this object is a service, find out what the
    // provider type is (the type used to look up the service)
    Type serviceType;
    if (Engine.Services.IsService(this, out serviceType))
    {
        // Tell the serializer about the provider type
        data.AddDependency(serviceType);

        // Set the data to the ServiceData
        sd.IsService = true;
        sd.Type = serviceType.FullName;
    }

    // Add the ServiceData to the SerializationData
    data.AddData("Component.ServiceData", sd);

    // Call the overridable function that allows components to provide data
    SaveSerializationData(data);

    return data;
}

// Reconstructs the Component from SerializationData
public void RecieveSerializationData(SerializationData Data)
{
    // Set the basic Component values
    this.DrawOrder = Data.GetData("Component.DrawOrder");
    this.Visible = Data.GetData("Component.Visible");
    this.Name = Data.GetData("Component.Name");

    // Get the ServiceData from the data
    ServiceData sd = Data.GetData("Component.ServiceData");

    // If the component was a service
    if (sd.IsService)
    {
        // Get the type back from the serializer
        Type t = Data.GetTypeFromDependency(sd.Type);

        // Add the service to the Engine
        Engine.Services.AddService(t, this);
    }

    // Set the owner GameScreen
    string parent = Data.GetData("Component.ParentScreen");
    this.Parent = Engine.GameScreens[parent];

    // Call the overridable function that allow components to load from data
    LoadFromSerializationData(Data);
}

// Overridable function to allow components to save data during serialization
public virtual void SaveSerializationData(SerializationData Data)
{
}

// Overridable function to allow components to load data during deserialization
public virtual void LoadFromSerializationData(SerializationData Data)
{
}

Finally, the Engine needs functions to save/load scenes from file:

// Save the current state of the engine to file
public static void SerializeState(string Filename)
{
    // Get the start time
    DateTime startTime = DateTime.Now;

    // Create an XmlWriter
    XmlWriterSettings set = new XmlWriterSettings();
    set.Indent = true;
    XmlWriter writer = XmlWriter.Create(new FileStream(Filename, FileMode.Create), set);

    // Create a Serializer
    Serializer s = new Serializer();

    // Write the start of the document, including the root node and save time
    writer.WriteStartDocument();
    writer.WriteStartElement("EngineState");
    writer.WriteAttributeString("Time", startTime.ToString());

    // Serialize the list of GameScreens
    s.WriteGameScreens(writer);

    // Write the component root node
    writer.WriteStartElement("Components");

    // Serialize all the components, if they want to be serialized
    foreach (GameScreen gameScreen in GameScreens)
        foreach (Component component in gameScreen.Components)
            if (component.Serialize)
            {
                writer.WriteStartElement(component.GetType().FullName);
                s.Serialize(writer, component.GetSerializationData(s, writer));
                writer.WriteEndElement();
            }

    // Finish the Components node
    writer.WriteEndElement();

    // Write out Assembly dependencies
    s.WriteDependencies(writer);

    // Finish the document
    writer.WriteEndElement();
    writer.WriteEndDocument();

    // Finish writing
    writer.Close();

    // Calculate elapsed time
    DateTime stopTime = DateTime.Now;
    TimeSpan elapsedTime = stopTime - startTime;
}

// Reload the state of the engine from file
public static void DeserializeState(string Filename)
{
    // Get the start time
    DateTime startTime = DateTime.Now;

    // Load the Xml document from file
    XmlDocument doc = new XmlDocument();
    doc.Load(Filename);

    // Locate the Components root node
    XmlNode ComponentsNode = doc.GetElementsByTagName("Components")[0];

    // Create a serializer
    Serializer s = new Serializer();

    // Reload the Assembly dependencies
    s.PopulateAssemblies(doc.GetElementsByTagName("Dependencies")[0]);

    // Deserialize each component in the file
    foreach (XmlNode node in ComponentsNode.ChildNodes)
        s.Deserialize(node);

    // Calculate the elapsed time
    DateTime stopTime = DateTime.Now;
    TimeSpan elapsedTime = stopTime - startTime;
}

And that’s all there is to the serialization system. Of course, we will now need to update most of our components to use it. I am not going to post these changes in the main body because they are so numerous. Instead, I will link to the updated versions of the components in another post that will be up shortly.

« SpriteBatch, Renderstates, and You Planet Google »