ScriptSystem - Loading Scripts
Prerequisites
- Implementing Scripting
- Embedding with Mono
- Scripts and Components - Mono
Article
The final piece of our scripting module architecturally is the ScriptSystem. After that, all that's left is writing the Mono-specific wrapper functions around engine functions we want to expose. That's going to be quite a bit of work, but it's largely repetitive. We'll start down that path in the next and final article of the series.
The ScriptSystem is the work horse of our architecture. It has 3 primary responsibilities:
- Keep track of the relationship between "local" (C++) and "remote" (C#) objects. We'll want to search both ways: given a C# object, find the C++ equivalent, and given a C++ object, find or create the C# equivalent.
- Load and keep track of the available script types. When we load a player or engine assembly (DLL), we'll need to go through all of the classes in it and see if they inherit from UMonoScript and can therefore be attached to entities via components.
- Convert types from C++ UVariants to Mono and C# types. When we write all of our wrapper functions, we'll be converting to and from MonoObjects all of the time, whether it's passing parameters or returning objects from C++ to C#.
While it may not seem like a lot, the ScriptSystem is a lot of code. In this Part I article, we'll be focusing primarily on #2 - how we load and process assembly files to treat C# class types as if they were native to our C++ engine.
Let's review our ScriptSystem base class:
class UScriptSystem : public USystem {
public:
UScriptSystem();
virtual ~UScriptSystem() = default;
/**
* Start system
*/
virtual void Start() { }
/**
* Update loop
*/
void Update(float delta);
/**
* Load library
*/
virtual void LoadScriptLibrary(std::string path) { }
/**
* Get or create an object relationship from Mono to local
*/
virtual UScriptObject* GetRemoteObject(std::string className, UObject* localObject) { return(0); }
/**
* Gets a local object, given a MonoObject
*/
virtual UScriptObject* GetLocalObject(void* remoteObject) { return(0); }
/**
* Find a script class
*/
virtual UScript* GetScript(std::string className) { return(0); }
protected:
// Fixed time step counter
float m_fixedUpdateCounter;
};This class is mostly virtual functions, which will be implemented by our Mono-specific implementation. You can see that it has a Start function, which will initialize any engine libraries and callback functions; it has a LoadScriptLibrary function which will load an assembly (DLL) into our domain; it has GetRemoteObject and GetLocalObject functions to help find our object relationships; and finally, it has a GetScript function, which will return a script that can be used in a component, which will search by class name.
The only function that the base ScriptSystem class implements is the Update function, which you saw in our article on Scripting Module Structure and is responsible for calling Update and / or FixedUpdate on the various components and scripts attached to entities in our game world:
void UScriptSystem::Update(float delta) {
std::vector<UScriptComponent*> scripts = UWorld::Instance()->GetComponents<UScriptComponent>();
auto it = scripts.begin();
for (; it != scripts.end(); it++) {
(*it)->Update(delta);
}
m_fixedUpdateCounter += delta;
float fixedUpdateStep = 1.0f / UENGINE_TICKS_PER_SECOND;
while (m_fixedUpdateCounter > fixedUpdateStep) {
it = scripts.begin();
for (; it != scripts.end(); it++) {
(*it)->FixedUpdate(fixedUpdateStep);
}
m_fixedUpdateCounter -= fixedUpdateStep;
}
}Let's look at our Mono specific implementation:
class UMonoScriptSystem : public UScriptSystem {
public:
UMonoScriptSystem() : m_monoDomain(0) { }
~UMonoScriptSystem() = default;
/**
* Start system
*/
void Start();
/**
* Load library
*/
void LoadScriptLibrary(std::string path);
/**
* Get or create an object relationship from Mono to local
*/
UScriptObject* GetRemoteObject(std::string className, UObject* localObject);
/**
* Gets a local object, given a MonoObject
*/
UScriptObject* GetLocalObject(void* remoteObject);
/**
* Find a script class
*/
UScript* GetScript(std::string className);
/**
* Convert a MonoObject to a Variant
*/
UVariant MonoObjectToVariant(MonoObject* object);
/**
* Handle array case
*/
MonoArray* VariantListToMonoArray(std::vector<UVariant> arr, std::string classHint = "");
/**
* Convert a variant to a Mono object
*/
MonoObject* VariantToMonoObject(UVariant variant, std::string classHint = "");
protected:
// Mappings from class types to C# class types
std::map<std::string, MonoClass*> m_monoClasses;
// Mono script types
std::map<std::string, UMonoScript*> m_monoScripts;
// Mono objects
std::map<UObject*, UMonoObject*> m_monoObjects;
std::map<MonoObject*, UMonoObject*> m_localObjects;
// Cached built-in Mono types
std::map<uint32_t, MonoClass*> m_cachedTypes;
// Mono domain
MonoDomain* m_monoDomain;
};The first 10 or so lines of the class look the same - we're just going to implement some of the base class functionality. However, you can see that we've added three new functions and a slew of properties. The functions are MonoObjectToVariant, VariantListToMonoArray, and VariantToMonoObject. As you can tell by the names, the first and last function convert Mono objects to UVariants and vice versa, while the VariantListToMonoArray function converts an std::vector of UVariants to a C# array.
We have 5 std::maps here for easy lookups. The first is to cache and find our MonoClass objects by class name. We'll use this pretty extensively when we call GetRemoteObject, which takes a class hint of what type to return if an object doesn't exist yet. The second map is specifically for scripts - that is, classes that inherit from UMonoScript and will override our 4 primary functions. We'll use this in our GetScript function to create components. Our next two maps are from UObjects to UMonoObjects and UMonoObjects to UObjects. These are our relationships from C++ to C# objects and vice versa. Finally, we have one more map, which caches the built-in MonoClass types - that is, the plain old data types used in C#: int, uint, string, bool, float, double, etc. We'll use these when we "box" and "unbox" values to pass back and forth. Finally, we have our MonoDomain.
Let's start at the top with the Start function:
void UMonoScriptSystem::Start() {
// If this is our first Mono library, create domain
if (m_monoDomain == 0) {
std::string libDir = "mono\\lib";
std::string configDir = "mono\\etc";
mono_set_dirs(libDir.c_str(), configDir.c_str());
mono_config_parse(NULL);
m_monoDomain = mono_jit_init("untitled-engine");
}
// Load our library
LoadScriptLibrary("UEngine.dll");
// Register callbacks
uscript_register_monomethods();
}The firs thing this function does is create and initialize our Mono domain. By default, when you install Mono, it will install into a shared directory. Program Files on Windows, the Library directory on Mac OS, etc. Instead, you'll want to package a specific version of Mono and the .NET framework with your application. From the installed versions, copy the "lib" and "etc" directories into your application's working directory. Here we've put them into a directory called "mono". You can then go into lib/mono and remove everything but the .NET version you want to use. You can even skinny it down further by only including certain C# libraries.
We tell Mono about all of this by using the mono_set_dirs function to specify a library directory and a config (etc) directory. We then call mono_config_parse(NULL) to use the default DLL maps, and finally, we call mono_jit_init, passing in our application / engine name to create a unique domain.
With Mono initialized, we call our LoadScriptLibrary function on our first library, called UEngine.dll. This assembly is the one we compile from all of our own C# classes that mirror our engine classes. We'll look at that function momentarily. Finally, we call a function called uscript_register_monomethods. This function is kind of a wrapper around all of our wrapper methods - it will call mono_add_internal_call for all of our exposed engine functions and point them to the appropriate wrapper. We'll dive into more detail in the next section, but here's a preview:
void uscript_register_monomethods() {
// UTransform
mono_add_internal_call("UEngine.UTransform::Move", uscript_utransform_move);
mono_add_internal_call("UEngine.UTransform::Rotate", uscript_utransform_rotate);
mono_add_internal_call("UEngine.UTransform::Scale", uscript_utransform_scale);
// ... many more setup calls here
}Each of the functions you see (uscript_utransform_move, uscript_utransform_rotate, and uscript_utransform_scale) are wrapper methods we write and then link to their C# class equivalent.
So the Start function: initialized our domain, loads our engine C# class library, and registers all of our callback functions. Let's start to look at the LoadScriptLibrary function. This function is about 200 lines long, so we'll break it down into logical parts. First, the easy stuff:
void UMonoScriptSystem::LoadScriptLibrary(std::string path) {
// Load assembly
MonoAssembly* assembly = mono_domain_assembly_open(m_monoDomain, path.c_str());
MonoImage* image = mono_assembly_get_image(assembly);
// If this is the first library, it should be ours
if (m_cachedTypes.empty()) {
m_cachedTypes[UVariant::VAR_BOOL] = mono_get_boolean_class();
m_cachedTypes[UVariant::VAR_INT32] = mono_get_int32_class();
m_cachedTypes[UVariant::VAR_INT64] = mono_get_int64_class();
m_cachedTypes[UVariant::VAR_UINT32] = mono_get_uint32_class();
m_cachedTypes[UVariant::VAR_UINT64] = mono_get_uint64_class();
m_cachedTypes[UVariant::VAR_FLOAT] = mono_get_double_class();
m_cachedTypes[UVariant::VAR_STRING] = mono_get_string_class();
m_cachedTypes[UVariant::VAR_VECTOR2] = mono_class_from_name(image, "UEngine", "Vector2");
m_cachedTypes[UVariant::VAR_VECTOR3] = mono_class_from_name(image, "UEngine", "Vector3");
m_cachedTypes[UVariant::VAR_VECTOR4] = mono_class_from_name(image, "UEngine", "Vector4");
m_cachedTypes[UVariant::VAR_QUATERNION] = mono_class_from_name(image, "UEngine", "Quaternion");
}The first few lines of this function open an assembly (DLL) and get the image. The image is the part of the DLL that contains executable .NET bytecode and will therefore have all of our classes and functions in it. Next, if we haven't loaded and cached our basic C# types, we'll do that. These are all pretty standard, except for the last 4. These are 4 classes that we will write but we want to be treated as "native" in our C# scripts, similar to how we use them in C++.
Next, we will iterate over all of the classes. For all of C#'s reflective capabilities, getting the information out of DLLs (or EXEs) is a challenge. We'll use some handy Mono functions to check the contents of our assembly:
// Load classes
const MonoTableInfo* table_info = mono_image_get_table_info(image, MONO_TABLE_TYPEDEF);
int rows = mono_table_info_get_rows(table_info);
for (int i = 0; i < rows; i++) {
MonoClass* _class = nullptr;
uint32_t cols[MONO_TYPEDEF_SIZE];
mono_metadata_decode_row(table_info, i, cols, MONO_TYPEDEF_SIZE);
const char* name = mono_metadata_string_heap(image, cols[MONO_TYPEDEF_NAME]);
const char* name_space = mono_metadata_string_heap(image, cols[MONO_TYPEDEF_NAMESPACE]);
_class = mono_class_from_name(image, name_space, name);
if (strcmp(name, "<Module>") == 0) {
continue;
}Inside of our assembly are various "tables" that give us more information about the contents of the file. We want the one that tells us about the "types" in the file - that is, the class types. Once we have a pointer to the table, we can iterate over the rows. In a typical table fashion, we iterate over one row at a time and get the columns, looking specifically at the name of this type and the namespace it is in. Since we're just looking at class types, we can then call mono_class_from_name to get the MonoClass object (saved here into _class). There is always one class called <Module> - we can skip that one.
Next, we really want to know if classes inherit from UMonoScript. To do that, we can use the very handy mono_class_is_subclass_of, which will even handle multiple levels of inheritance. To see if something inherits from UMonoScript, we need the MonoClass object for the UMonoScript C# class, so the next block of code checks if we have it cached already, and if not, tries to find it by name:
// Get UScript class
MonoClass* scriptClass = 0;
auto sit = m_monoClasses.find("UMonoScript");
if (sit != m_monoClasses.end()) {
scriptClass = sit->second;
}
if (scriptClass == 0) {
// Try to load from current library
scriptClass = mono_class_from_name(image, name_space, "UMonoScript");
m_monoClasses["UMonoScript"] = scriptClass;
}
UASSERT(scriptClass != 0, "Unable to find script base class.");Now we're into the real meat of why we're here: we can check if this particular class that we're looking at in our loop inherits from UMonoScript. If it does, we know it may override the Initialize, Update, FixedUpdate, and Destroy functions we saw in our Scripts and Components article:
// Check whether this class inherits from UScript
int classIDCounter = 10;
if (mono_class_is_subclass_of(_class, scriptClass, false)) {
// Ignore the UMonoScript class itself
if (_class == scriptClass) continue;
// If so, need to store specific function references
UMonoScript* script = new UMonoScript();
script->className = name;
// Register with MetaSystem
GetUSystem<UMetaSystem>()->RegisterType<UMonoScriptComponent>(name, 50000 + classIDCounter);
classIDCounter += 10;
// Find functions - Initialize
MonoMethod* method = mono_class_get_method_from_name(_class, "Initialize", 0);
if (method) {
script->initFunc = (UMonoScriptInitFunction)mono_method_get_unmanaged_thunk(method);
}
// Update
method = mono_class_get_method_from_name(_class, "Update", 1);
if (method) {
script->updateFunc = (UMonoScriptUpdateFunction)mono_method_get_unmanaged_thunk(method);
}
// FixedUpdate
method = mono_class_get_method_from_name(_class, "FixedUpdate", 1);
if (method) {
script->fixedUpdateFunc = (UMonoScriptFixedUpdateFunction)mono_method_get_unmanaged_thunk(method);
}
// Destroy
method = mono_class_get_method_from_name(_class, "Destroy", 0);
if (method) {
script->destroyFunc = (UMonoScriptDestroyFunction)mono_method_get_unmanaged_thunk(method);
}From the top, you can see that we initialize a counter. If you read our Reflection article, you'll know that we need to register all class types with the MetaSystem for the UType() function to work properly on all of our UObjects. We'll use this counter to add all of our C# class types to our UMetaSystem so that they can be treated as first-class citizens in C++ as well.
Next, we check that this class inherits from UMonoScript and then that it isn't the UMonoScript class itself (because we already have that class from the previous block of code). Now, knowing this is a UMonoScript, we create a new UMonoScript C++ object and set the className. We also register the class type with our UMetaSystem - the 50000 here is arbitrary and just ensures that the other C++ classes don't catch up. We increment by 10, so we would need 5000 C++ classes to have an overlap, which feels safe for now.
Finally, we see if this class implements our override functions by calling mono_class_get_method_from_name and passing the class we're looking at (_class), the name of the function we want to find, and the number of parameters it will have. If we get a non-NULL response, we then call mono_method_get_unmanaged_thunk to get a C++ function pointer we can use to call into our C# code. As seen in the Scripts and Components article, our UMonoScriptComponent, once attached to an Entity, gets the C# Initialize function called on creation and the C# Update and FixedUpdate functions called during the game loop.
With our function pointers in place, the last thing we want to know about is which public variables of this class have been tagged with the SerializeField attribute and should be serialized and deserialized with the components they are attached to. We can use the mono_class_get_fields function to iterate over our class fields:
// Find public serialized variables
void* iter = NULL;
MonoClassField* field = mono_class_get_fields(_class, &iter);
while (field != NULL) {
std::string fieldName = mono_field_get_name(field);
MonoCustomAttrInfo* info = mono_custom_attrs_from_field(_class, field);
// If info is null, no attributes
if (info == 0) {
field = mono_class_get_fields(_class, &iter);
continue;
}
// Check for custom attribute
bool serializable = false;
for (int i = 0; i < info->num_attrs; i++) {
MonoMethod* attrmethod = info->attrs[i].ctor;
std::string attrName = mono_class_get_name(mono_method_get_class(attrmethod));
if (attrName == "SerializeField") serializable = true;
}
if (serializable == false) {
field = mono_class_get_fields(_class, &iter);
continue;
}
// Check field accessibility
uint32_t flags = mono_field_get_flags(field) & MONO_FIELD_ATTRIBUTE_FIELD_ACCESS_MASK;
if ((flags & MONO_FIELD_ATTRIBUTE_PUBLIC) == false) {
field = mono_class_get_fields(_class, &iter);
continue;
}
// For public fields, map type
MonoType* fieldType = mono_field_get_type(field);
int enumType = mono_type_get_type(fieldType);
int variantType = 0;
switch (enumType) {
case MONO_TYPE_STRING:
case MONO_TYPE_CHAR:
variantType = UVariant::VAR_STRING;
break;
case MONO_TYPE_CLASS:
variantType = UVariant::VAR_OBJECT;
break;
case MONO_TYPE_BOOLEAN:
variantType = UVariant::VAR_BOOL;
break;
case MONO_TYPE_I4:
variantType = UVariant::VAR_UINT32;
break;
case MONO_TYPE_U4:
variantType = UVariant::VAR_INT32;
break;
case MONO_TYPE_I8:
variantType = UVariant::VAR_INT64;
break;
case MONO_TYPE_U8:
variantType = UVariant::VAR_UINT64;
break;
case MONO_TYPE_R4:
case MONO_TYPE_R8:
variantType = UVariant::VAR_FLOAT;
break;
default:
break;
}
UASSERT(variantType != 0, "Unable to map variant type.");
script->vars[fieldName] = variantType;
field = mono_class_get_fields(_class, &iter);
}The mono_class_get_fields takes as parameters the class type (_class) and an iterator, which is really a void pointer. Once the void pointer is null, then there are no more fields left to inspect.
The first two lines of our loop get the field name and get any custom attributes. If the returned pointer to custom attributes is zero, then the variable has none. Again, this is different behavior than say Unity, which serializes all public variables and any private variables tagged with SerializeField - we are only going to serialize public variables that are also explicitly tagged with the SerializeField attribute.
Remember from our Embedding with Mono article that all attributes are actually also C# classes. When you tag something with an attribute in C#, you're actually calling the constructor of the class. The next block iterates through each attribute (if there is more than one), and for each one we only have the constructor, which is a MonoMethod object, so we call mono_method_get_class to get the MonoClass object and then check the name of that class (using mono_class_get_name) against what we're expecting (SerializeField). If we find a SerializeField attribute, then we set our "serializable" variable to true. Of course, if we exit the loop and serializable is false, then we iterate on to the next field.
Finally, we check the accessibility of our variable to ensure it is public. This is done using the mono_field_get_flags function on the field. Remember that the constants in this block of code are our own - see the Embedding with Mono article for their values and why we need them.
If the field is public and has the SerializeField attribute, then what we really want to know if what type it is. We'll use this later in the editor to allow the user to input and change the values of the C# variables inside the editor.
We call mono_field_get_type on the field to get a MonoType object, which we then pass into mono_type_get_type to get a more usable enumerated type. The switch block and case statements simply map the Mono enum types to our UVariant types. Finally, we store the serialized variable name and type into our script. Then we store our UMonoScript types into our script cache, and all class types into our MonoClass cache:
// Add script type
m_monoScripts[name] = script;
}
// Add to list
m_monoClasses[name] = _class;
}
}Conclusion
We can now load DLLs with C# classes into our engine and treat those classes as first-class citizens - especially where they are inherited from UMonoScript. We can get function pointers to call C# functions from C++ code in our components, and we can check class fields or variables to see if they should be serialized into our editor, storage system, networking, and more.
In the next article, we'll finish off our ScriptSystem and cover off managing local and remote objects, as well as conversions from UVariant types to C# types and vice versa.