Variants - Generic Containers
Prerequisites
- The UObject Base Class
Article
If you've ever used Qt, you've probably come across the QVariant class. If you've ever used Epic's Unreal Engine, you may have seen the TVariant. Boost has the Boost.Variant, and even C++17 has adopted the std::variant.
In essence, a variant is a way to pass around data that you don't really know or care what type it is until it's time to use it. In practice, it is usually a union of many different types, of which only one is actually used for storage. Let's take a look at a very simple example:
union {
int32_t i32;
uint32_t ui32;
int64_t i64;
uint64_t ui64;
bool b;
float f1;
char* str;
void* ptr;
UObject* obj;
} Value;This union can be a 32-bit or 64-bit signed or unsigned integer, a boolean value, a floating point number, a string, a void pointer, or a pointer to an object that inherits from our base UObject class. Note that these can only be POD (plain old data) types - you cannot use an std::string for example. The actual value of the variable is determined on assignment, for example:
Value i = 32.0f; // i is now a float with the f1 property set to 32Presumably, at the time of use, you would know this was a floating point value, and therefore reference i.f1 in your code. But what if you didn't know? What if we wrapped a class around this and used C++ constructors and properties to create a generic container that could tell us what type it is?
To do that, we would need to store an additional field: the type, as it is set. Let's take a look:
// Supported types
enum Type {
VAR_INT32 = 1, // 1
VAR_UINT32, // 2
VAR_INT64, // 3
VAR_UINT64, // 4
VAR_BOOL, // 5
VAR_FLOAT, // 6
VAR_VECTOR2, // 7
VAR_VECTOR3, // 8
VAR_VECTOR4, // 9
VAR_QUATERNION, // 10
VAR_STRING, // 11
VAR_VOID, // 12
VAR_OBJECT, // 13
};Now, a combination of the Value and the Type tell us what kind of data we are storing and which property of our union we should use to access it.
You'll also see that I've added some new types that are typical of a game engine: vectors of size 2, 3, and 4 components, and a quaternion. We can nest our unions together to create additional types. I've included a simple example here that stores 4 components for all vectors, but you could create vec2, vec3, and vec4 nested unions to save some space.
Let's start putting these together into a class:
class UVariant {
public:
UVariant();
virtual ~UVariant();
// Supported types
enum Type {
VAR_INT32 = 1, // 1
VAR_UINT32, // 2
VAR_INT64, // 3
VAR_UINT64, // 4
VAR_BOOL, // 5
VAR_FLOAT, // 6
VAR_VECTOR2, // 7
VAR_VECTOR3, // 8
VAR_VECTOR4, // 9
VAR_QUATERNION, // 10
VAR_STRING, // 11
VAR_VOID, // 12
VAR_OBJECT, // 13
};
struct Value {
union {
int32_t i32;
uint32_t ui32;
int64_t i64;
uint64_t ui64;
bool b;
float f1;
char* str;
void* ptr;
UObject* obj;
union {
float x;
float y;
float z;
float w;
} vec;
};
};
/**
* Constructors
*/
UVariant(int32_t value);
UVariant(uint32_t value);
UVariant(int64_t value);
UVariant(uint64_t value);
UVariant(bool value);
UVariant(float value);
UVariant(char* value);
UVariant(glm::vec2 value);
UVariant(glm::vec3 value);
UVariant(glm::vec4 value);
UVariant(glm::quat value);
UVariant(std::string value);
UVariant(UObject* value);
UVariant(const UVariant& value);
/**
* Set operators
*/
UVariant& operator =(int32_t rhs);
UVariant& operator =(uint32_t rhs);
UVariant& operator =(int64_t rhs);
UVariant& operator =(uint64_t rhs);
UVariant& operator =(bool rhs);
UVariant& operator =(float rhs);
UVariant& operator =(char* rhs);
UVariant& operator =(glm::vec2 rhs);
UVariant& operator =(glm::vec3 rhs);
UVariant& operator =(glm::vec4 rhs);
UVariant& operator =(glm::quat rhs);
UVariant& operator =(std::string rhs);
UVariant& operator =(UObject* rhs);
UVariant& operator =(const UVariant& rhs);
protected:
// Type
int m_type;
// Value data
Value m_data;
}These functions will make up the basis of our Variant class. In particular, each constructor and assignment operator pair allows us to create variants from those types that we have allowed. For example, we can now write Variant v = glm::vec3(0, 1, 0); and store a vector into our variant class. While we won't go through every example, let's take a look at a few of them to see how they work:
UVariant::UVariant(uint32_t value) {
*this = value;
}
UVariant::UVariant(bool value) {
*this = value;
}
UVariant::UVariant(vector2 value) {
*this = value;
}First, we have our constructors. Those just pass the work off to the assignment operators (every constructor has the same body):
UVariant& UVariant::operator =(int32_t rhs) {
m_type = VAR_INT32;
m_data.i32 = rhs;
m_size = sizeof(int32_t);
return *this;
}
UVariant& UVariant::operator =(bool rhs) {
m_type = VAR_BOOL;
m_data.b = rhs;
m_size = sizeof(bool);
return *this;
}
UVariant& UVariant::operator =(glm::vec2 rhs) {
m_type = VAR_VECTOR2;
m_data.vec.x = rhs.x;
m_data.vec.y = rhs.y;
m_size = sizeof(float) * 2;
return *this;
}Now we start to get into the meat of our class. In each assignment operator, we set the type and the size, as well as the relevant properties of our union. In the case of an unsigned 32-bit integer, we set the i32 property; for a boolean value, we set the b property; and for a vector with two components, we set the vec.x, and vec.y properties.
Since our Variant is now aware of its own type, we can also add some functions to determine whether it is a specific type, and to get the value if it is that type. Some code will help that make more sense. In our class, we can add functions that check for specific types, and functions that return those types:
class UVariant {
public:
// All of the stuff from above here...
/**
* Checkers
*/
bool IsInt();
bool IsUInt();
bool IsInt64();
bool IsUInt64();
bool IsBool();
bool IsFloat();
bool IsNumeric();
bool IsVector2();
bool IsVector3();
bool IsVector4();
bool IsQuaternion();
bool IsString();
bool IsObject();
bool IsNull();
bool IsVariant() { return true; }
/**
* Get operators
*/
int32_t AsInt();
uint32_t AsUInt();
int64_t AsInt64();
uint64_t AsUInt64();
bool AsBool();
float AsFloat();
vector2 AsVector2();
vector3 AsVector3();
vector4 AsVector4();
quaternion AsQuaternion();
std::string AsString();
UObject* AsObject();
UVariant* AsVariant();
template<class T> T AsObject() {
if (m_type == VAR_OBJECT) {
return(dynamic_cast<T>(m_data.obj));
}
return(0);
}
}In a moment, we'll take a look at a few implementations. I did want to call attention to the templated AsObject function included above. As mentioned in our UObject article, (almost) all objects in our engine are going to be derived from UObject. This templated function allows us to cast those objects up or down into their various types. For example, if we wanted to check if this was a System, we could simply call IsObject() to confirm it is a UObject, and then AsObject< System > to return a generic system object on which we know we could call our Initialize, Update, or Shutdown functions.
Let's take a look at a few other implementations... first, our checkers:
bool UVariant::IsInt() {
return(m_type == VAR_INT32;
}
bool UVariant::IsBool() {
return(m_type == VAR_BOOL);
}
bool UVariant::IsVector2() {
return(m_type == VAR_VECTOR2);
}And finally, our "getters":
int32_t UVariant::AsInt() {
if (m_type == VAR_INT32) {
return m_data.i32;
return(0);
}
bool UVariant::AsBool() {
if (m_type == VAR_BOOL) {
return(m_data.b);
}
return(false);
}
vector2 UVariant::AsVector2() {
if (m_type == VAR_VECTOR2) {
return vector2(m_data.vec.x, m_data.vec.y);
}
return(vector2(0, 0));
}All together, we can now create some very powerful snippets of code. Variants can now be integers, floating point variables, booleans, vectors, quaternions, objects, and more. As we pass them around our engine as Variant objects, the function calls and implementations do not need to worry about what data is actually stored in them. We can discover their data type and return them in their original format - for objects, we can even upcast or downcast them along their parent and subclass types, depending on how specific we want to be.
While the variant is a powerful tool already, it really comes in handy in three places:
- Integration with scripting engines - whether you're using Python, V8 for JavaScript, Mono for C#, or Lua, most scripting engines have generic types that they will pass back and forth to your engine. Integrating and translating between those and your engine's native data formats is a perfect use case for the variant class. For example, V8 has a v8::Value that looks and feels like the variant class above that defines functions such as IsBoolean and ToBoolean. Mapping those to a variant that can be passed around your engine's internals is made much easier with our own Variant implementation.
- Serialization - when storing a list of key-value pairs to be serialized, the values are often different types. Now all values can simply be a Variant, and we can let the various serialization methods figure out how they're going to store our supported data types. JSON may store a vector as {x: 0.0, y: 1.0, z: 0.0}, while storing a 3D vector in SQLite may be a string "(0, 1, 0)". Both serializers can check the IsVector3() method, get the value with AsVector3() and then serialize the values however they need. When we create an editor for our engine, we'll also use Variants to help us create dynamic user interfaces (that show a checkbox for a Boolean or 3 text fields for the X, Y, and Z of a vector).
- Reflection - One popular feature of many reflection systems is the ability to call functions with a key-value list of parameters. For example, our System class has an Update() function that takes a single float (delta time between frames). If we supported reflection, you could do something like
system->Call("Update", [(float)delta]);to call the same function by name and pass in the list of parameters as an array. This approach also plays into our integration with scripting engines, where some (like V8/JavaScript) are much less structured up-front about the functions that can be run on any object.
Internally, we won't use the Variant much, except for serialization. But as we begin to get more sophisticated with our scripting libraries and reflection systems and try to give our game designers and developers more tools, the Variant class is an invaluable tool for abstracting data that we will use extensively.