Getting started with SpiderMonkey to run JavaScript in Delphi

	Getting started with SpiderMonkey to run JavaScript in Delphi

Run JavaScript in Delphi

Introduction

Some time ago, I worked on integrating the PascalScript engine into TaskRunner. This is an open-source build automation tool, which is available on GitHub, see the Task Runner repository

The scripting is necessary to customize the building process. PascalScript provides wide opportunities. However, there is the other scripting language, JavaScript, which can be considered as a more powerful and flexible tool for implementing specific building and deployment tasks. There is a huge number of modules, extensions, and frameworks, which allow you to solve a wide range of tasks with the ability to scale solutions

Therefore, the next step in the TaskRunner development was to add the JavaScript support. Before we get started searching for a suitable engine, let's list the requirements:

So, to make things working, I had to solve the following tasks:

The possibility to transfer data from a script to Delphi, and back;
Working with the file system, writing, and reading text files. The ActiveX technology and it's FileSystemObject are deprecated, so fs Node.js is preferable;
Running scripts in threads.

Nowadays, there are lots of scripting engines, including ones with open-source code. You can read an overview of different scripting engines in the following article: Delphi JavaScript execution - ditch TWebBrowser for ChakraCore

This is possible the best overview I've ever read about JavaScript integrations in Delphi. The V8 engine is rather powerful. However, has some problems integrating with Delphi. Besen represents 100% Delphi implementation. But it also has known performance issues. The Microsoft ChakraCore library and an open-sourced Delphi/FPC binding look very promising. But in my quick attempts, I couldn't use Node.js modules in scripts.

Finally, the choice was made in favor of SpiderMonkey. This very powerful engine is a Mozilla product, and is included in the FireFox web browser.

Thus, we have the following tasks:

Installing SpiderMonkey;
Writing a test Delphi app, which runs a test JavaScript code;
Testing the engine in threads.

Installing SpiderMonkey

The SpiderMonkey engine has a very good integration with Delphi, which is implemented in the Synopse mORMot project.

A brief study of demos and tests, which are available in their GitHub repository, gives us good usage examples. You can declare classes in Delphi, create, and use instances of these classes in a script, as well as transfer data from Delphi to a script, and back using class properties and methods. It looks like this is what we need. In addition, this scripting engine allows you to debug scripts. But let's keep the script debugging for the future article.

SpiderMonkey, as every open-source product, requires some work for installing and using. In addition, mORMot requires a patched version of SpiderMonkey. The patches concern external procedure declarations: mORMot uses extern "C" for all imported DLL functions.

You can learn more in the mORMot installation instructions, which are available in their GitHub repository: SpiderMonkey build instructions

There are two ways to get started. You can download the SpiderMonkey sources, patch them, and compile on your side. You must be experienced in Mozilla Build and have the time to set up numerous configurations. The other way is to download precompiled SpiderMonkey DLLs using the links from the Synopse website. I used SpiderMonkey 52, as recommended by Synopse: SpiderMonkey downloads

However, there is one issue: the website, which contains the DLL archives, is not available from everywhere. If you are in a “wrong” location, like me, you will unable to download DLLs. The solution is simple: don't doubt to specify some proxy in your web browser.

Next, you need to download the SyNode sources - a Delphi integration library for SpiderMonkey. SyNode is a part of the mORMot project, and is available on GitHub, see the mORMot repository

At the time of writing this article, the master branch experienced difficulties when compiling the project. Therefore, let's use the sources from the latest stable release instead.

Writing and running a test app

Among the demos available within the mORMot repo, there are ones, which do everything we need. Let’s take a look at the SpiderMonkey45Binding.dproj project.

This project registers global objects, transfers data from Delphi using object properties, calls Delphi functions from a script, and loads Node.js modules. In particular, the example shows how to use the "fs", "path", and "http" modules. There is also an example of a custom module, which is loaded from DLL. The sources for this DLL module are available in the repository, as well. But in my first attempt, this module didn't not work. I decided not to dig out the problem and focused on the listed above tasks, required to incorporate the engine into TaskRunner.

Let's create a new Delphi project and add all necessary parts from the mentioned demo. You must specify paths to the moRMot source folders. There are two important classes, TSMEngineManager and TSMEngine, which are necessary to work with this scripting engine.

Next, create a new Delphi class, and make it available from a script. The class methods, which are necessary in a script, must be declared within the "published" section. This allows the mORMot runtime to automatically register these methods in the scripting engine. The created Delphi class must be enclosed in the {$M+} compiler option. As an alternative, you can inherit your class from TPersistent.

// [Delphi]
{$M+}
   TjobParams = class
   private
      Fdata: Tstrings;
   public
      constructor Create;
      destructor Destroy; override;

      property Data: Tstrings read Fdata;
   published
      function setValue(cx: PJSContext; argc: uintN; var vp: JSArgRec): Boolean;
      function getValue(cx: PJSContext; argc: uintN; var vp: JSArgRec): Boolean;
   end;
{$M-}

In order to use the same class instance within different threads, we need to protect the FData field using a critical section - paramAccessor: TCriticalSection. This allows us to avoid a thread race condition in the future.

// [Delphi]
function TJobParams.getValue(cx: PJSContext; argc: uintN; var vp: JSArgRec): Boolean;
begin
   if (argc <> 1) or (not vp.argv[0].isString) then
   begin
      raise ESMException.Create('getValue accepts one string parameter');
   end;

   paramAccessor.Enter();
   try
      vp.rval := SimpleVariantToJSval(cx, FData.Values[vp.argv[0].asJSString.ToString(cx)]);
   finally
      paramAccessor.Leave();
   end;

   Result := true;
…
end;

// [Delphi]
function TJobParams.setValue(cx: PJSContext; argc: uintN; var vp: JSArgRec): Boolean;
begin
   if (argc <> 2) then
   begin
      raise ESMException.Create('setValue accepts two string parameters');
   end;

   if not (vp.argv[0].isString and vp.argv[1].isString) then
   begin
      raise ESMException.Create('setValue parameters should be strings');
   end;

   paramAccessor.Enter(); //allows to avoid the thread racing condition
   try
      FData.Values[vp.argv[0].asJSString.ToString(cx)] := vp.argv[1].asJSString.ToString(cx);
   finally
      paramAccessor.Leave();
   end;

   Result := true;
…
end;

The registration of the class as a global object occurs in the TSMEngineManager.OnNewEngine event handler. The created TSMEngine instance requires a name. So, a script debugger will have the ability to connect to the engine for tracing.

// [Delphi]
procedure TForm1.DoOnCreateNewEngine(const aEngine: TSMEngine);
begin
   aEngine.defineClass(FJobParams.ClassType, TSMSimpleRTTIProtoObject, aEngine.GlobalObject);
   aEngine.GlobalObject.ptr.DefineProperty(aEngine.cx, 'jobParams',
      CreateJSInstanceObjForSimpleRTTI(aEngine.cx, FJobParams, aEngine.GlobalObject),
      JSPROP_ENUMERATE or JSPROP_READONLY or JSPROP_PERMANENT
   );
end;

// [Delphi]
function TForm1.DoOnGetEngineName(const aEngine: TSMEngine): RawUTF8;
begin
Result := 'FormEngine';
end;

In the OnFormCreate event when the application starts, we create a manager. The engine is accessed through the manager, namely through FSMManager.ThreadSafeEngine().

// [Delphi]
procedure TForm1.FormCreate(Sender: TObject);
begin
   FJobParams := TJobParams.Create();
   FJobParams.Data.Values['name1'] := 'value1'; //set up initial data for jobParams

   FSMManager := TSMEngineManager.Create(''); //because we use Node.js modules from a .res file.
   FSMManager.MaxPerEngineMemory := 512 * 1024 * 1024;
   FSMManager.OnNewEngine := DoOnCreateNewEngine;
   FSMManager.OnGetName := DoOnGetEngineName;

   FEngine := FSMManager.ThreadSafeEngine(nil);
end;

// [Delphi]
procedure TForm1.FormDestroy(Sender: TObject);
begin
   FSMManager.ReleaseCurrentThreadEngine();
   FSMManager.Free();
   FJobParams.Free();
end;

A test JavaScript will look as follows:

// [JavaScript]
var s1 = jobParams.getValue("name1");
s1 += "_updated";
jobParams.setValue("name1", s1);
var s2 = jobParams.getValue("name1");

var fs = require('fs');
fs.writeFileSync("hello.txt", "\ufeff" + s2);

We can add BOM (\ ufeff) at the beginning of the text to indicate that the file uses Unicode, as well as to indirectly indicate the encoding.

The script runs using the TSMEngine.Evaluate method. You must specify a name for the script. We will show the process of data transferring between Delphi and JavaScript using the registered global object. The script reads data from the global object, modifies, and returns back to the object. Additionally, the script saves data to a file using the "fs" module from Node.js.

// [Delphi]
procedure TForm1.btnEvaluateClick(Sender: TObject);
var
   res: jsval;
begin
   FEngine.Evaluate(memSource.Lines.Text, 'script.js', 1, res);
   ShowMessage('Done: ' + FJobParams.Data.Text);//modified data are shown here
end;

The script works like a charm. It's time to test the code in threads. To do this, let's create the TScriptThread class - a TThread descendant.

// [Delphi]
TScriptThread = class(TThread)
private
   FJobParams: TJobParams;
   FSMManager: TSMEngineManager;
   FScript: string;

   procedure DoOnCreateNewEngine(const aEngine: TSMEngine);
   function DoOnGetEngineName(const aEngine: TSMEngine): RawUTF8;
protected
   procedure Execute; override;
public
   constructor Create(const AScript: string; AJobParams: TJobParams);
end;

The SyNode integration assumes one single TSMEngineManager and multiple TSMEngine instances. Due to the TaskRunner's implementation specific, we cannot share the same instance of TSMEngineManager between running jobs. Therefore, we have to create a new instance of TSMEngineManager along with TSMEngine per each thread. Theoretically, this may cause performance issues. However, this decision allows us to quickly integrate SpiderMonkey into TaskRunner with minimum coding. So, the test code will look as follows:

// [Delphi]
procedure TForm1.btnMultiThreadEvaluateClick(Sender: TObject);
var
   thread: TThread;
   script: string;
   i, count: Integer;
begin
   script := memSource.Lines.Text;
   count := edtThreadCount.Value;

   for i := 0 to count - 1 do
   begin
      //this simple trick requires the hello.txt name in the javaScript source and allows each thread to use its own filename
      thread := TScriptThread.Create(StringReplace(script, 'hello.txt', 'hello' + IntToStr(i) + '.txt', [rfIgnoreCase, rfReplaceAll]), FJobParams);
      thread.FreeOnTerminate := True;
      thread.Start();
   end;

   ShowMessage(Format('%d concurrent threads were started.', [count]));
end;

Conclusion

You can download the source code for all the examples in this article on GitHub

SpiderMonkey is a great engine, which can do all the tasks we need. Now, we are actively working on adding SpiderMonkey to the TaskRunner app. I think, when this article will be published, we will release a new version of this build automation tool, as well.

Feel free to subscribe to our Email list

Nikita Shirokov
Clever Components team
www.clevercomponents.com