Tag Archives: multithreading

Multithreading VBA using VBscript

1 Star2 Stars3 Stars4 Stars5 Stars (1 votes, average: 5.00 out of 5)
Loading...

Recently I published the post on using C#.NET to do multithreading in VBA – find it here: EXCEL: Multithreading in VBA using C#.NET. In that post I stated that multithreading is not possible natively in VBA only using external libraries e.g. COM/dll in C#.NET. In result of that post my attention was drawn to an idea I once thought about – using the Windows Shell to execute VBscript. Daniel Ferry’s article looks into it and gives a great example of how you can actually achieve multithreading in VBA using a VBscript swarm without resorting to external libraries although still reaching out of native VBA.

I strongly encourage you to visit Daniel’s article page and download his fantastic example of website scraping using a VBscript swarm vs. singlethreaded VBA IE / MSXML2 scraping. I loved the animation showing how the swarm runs!

However, in this post I wanted to present you with a basic example and provide you with some reusable code that you might use for your own purposes to implement multithreading, w/o resorting to external libraries. Next I will take a look at the increase in performance which I achieved on a simple algorithm due to multithreading.

Multithreading VBA using VBscript example

So let’s start with the example:

Public Declare Sub Sleep Lib "kernel32" (ByVal dwMilliseconds As Long)
Dim maxThreads As Long, divTabSize As Long, startTime As Double
Public Sub MultithreadedMultiplication()
    '---Clear previous values---
    If Range("A2").Value <> "" Then
        Range("A2:A" & Range("A1").End(xlDown).Row).Value = ""
    End If
    
    Dim thread As Long
    maxThreads = 4
    divTabSize = 10000 'Amount of elements in the division table
    'A simple division algorithm (a multiplication table would quickly cause overflow)
    startTime = Timer
    For thread = 1 To maxThreads
        CreateVBScriptThread thread, maxThreads, divTabSize
    Next thread
    CheckIfFinished 'Wait until all threads finish
End Sub
Public Sub CreateVBScriptThread(threadNr As Long, maxThreads As Long, divTabSize As Long)
    Dim s As String, sFileName As String, wsh As Object
    '---Copy parameters to VBscript---
    s = s & "dim i, j, oXL, x, startTime, endTime" & vbCrLf
    s = s & "startTime = Timer" & vbCrLf
    '---Compute partition of the division table---
    s = s & "For i = " & CInt(CDbl(divTabSize) * (threadNr - 1) / maxThreads + 1) & " To " & CInt(CDbl(divTabSize) * threadNr / maxThreads) & vbCrLf
    s = s & "   For j = 1 to " & divTabSize & vbCrLf
    s = s & "      x = i / j" & vbCrLf
    s = s & "   Next" & vbCrLf
    s = s & "Next" & vbCrLf
    '---Save the threadNr back to your Excel file! - this is the incredible part really
    s = s & "Set oXL = GetObject(, ""Excel.Application"")" & vbCrLf
    s = s & "endTime = Timer" & vbCrLf
    s = s & "oXL.workbooks(""" & ThisWorkbook.Name & """).sheets(""Status"").Range(""" & "A" & (threadNr + 1) & """) = (endTime - startTime)" & vbCrLf
    '---Save the VBscript file---
    sFileName = ActiveWorkbook.Path & "Thread_" & threadNr & ".vbs"
    Open sFileName For Output As #1
    Print #1, s
    Close #1
    '---Execute the VBscript file asynchronously---
    Set wsh = VBA.CreateObject("WScript.Shell")
    wsh.Run """" & sFileName & """"
    Set wsh = Nothing
End Sub
'---------Verify if finished----------
Sub CheckIfFinished()
    Dim endTime As Double, elapsed As String
    Do Until False
        DoEvents
        If Range("A2").Value <> "" Then
            If Range("A2:A" & Range("A1").End(xlDown).Row).Count = maxThreads Then
                endTime = Timer
                elapsed = "" & Format(endTime - startTime, "0.00")
                Range("F1").Offset(maxThreads).Value = maxThreads
                Range("G1").Offset(maxThreads).Value = elapsed
                Exit Sub
            End If
        End If
        Sleep 100
    Loop
End Sub

Yeah, that’s basically it! Just 40-something lines of code for a simple multithreading example in VBA. I would have thought that it would be much more complicated, but apparently as you can see this is not the case.

The example code runs a very simple algorithm which aim is to calculate a division table (just to avoid multiplication overflow). If you use more than one thread the division table will be partitioned so each thread gets an equal slice of the action. Each VBscript will run in a separate thread, calculate the partition and simply return it’s run time directly to our current Excel Workbook. The result will look somewhat like this:

Multithreading in VBA using VBscript
Multithreading in VBA using VBscript

The basic approach is the following:

  • Take any algorithm and partition it for each separate thread
  • Create as many VBscript files and you need threads
  • Execute each one concurrently using the WScript.Shell object

As mentioned each VBscript thread, if required, can save it’s output directly to the current Excel Worbook which is incredible and actually increases the practicality of this example.

Now for comparison I could right the same algorithm in native VBA:

Sub DivisionSingleThread()
    Dim i, j, x, startTimeS, endTimeS
    startTimeS = Timer
    '---Compute partition of the division table---
    For i = 1 To divTabSize
       For j = 1 To divTabSize
          x = i / j
       Next
    Next
    endTimeS = Timer
    Range("A2") = (endTimeS - startTimeS)
    elapsed = "" & Format(endTimeS - startTimeS, "0.00")
    Range("G1").Offset(maxThreads).Value = elapsed
End Sub

Performance analysis

Now what about performance. You would expect multithreading to significantly increase your algorithms performance. I tested the algorithm with a 5000 x 5000 table of elements (5000×5000 divisions) on a 2-core 4 threaded Intel processor (i5-4300U) with the following results:

VBA vs. VBscript
VBA vs. VBscript

To be frank I was very surprised to see that the singlethreaded VBA algorithm was actually faster than any VBscript multithreaded example. The VBscript algorithm reached its highest improvement in terms of performance at 2 VBscript threads. Adding additional threads did not seems to significantly improve performance, although I was expecting more or less to improve execution time proportionally until 4 threads are reached. This is probably because my processor is only 2-core and the additional threads are virtual.

To make the 2 algorithms as similar as possible I did not explicitly declare my variable types which Michael from VBA4All rightfully noticed (in my previous post) can help significantly improve VBA performance. If I would modify the VBA algorithm accordingly the execution time would drop to even approx. 1 second! Unfortunately VBscript forbids explicit variable data type declaration which is a serious setback.

Does this mean that multithreading in VBA using VBscript does not make sense? In most cases it won’t. However, there are cases when this could be useful e.g. in Daniel VBscript swarm web scraping example where VBscript utilizes similar objects/system resources as VBA. What is more, although VBscript was much slower on my 2-core laptop, if I had much more cores/processors VBscript might actually make up the distance.

Let me know what you think!

Be sure to checkout a comparison of all available approaches to VBA multithreading:
EXCEL: Multithreading VBA – VBA vs. VBscript vs. C#.NET.

Find the example XLSM file here:

Multithreading VBA using C#.NET

1 Star2 Stars3 Stars4 Stars5 Stars (4 votes, average: 4.50 out of 5)
Loading...

Multithreading in VBA in Excel is possible! No wait… that can’t be true right? Almost everyone knows that VBA runs in a single thread. So am I kidding? Not quite! Although this may seems like a workaround there is actually a way to multithread your VBA algorithms. There is only 1 condition… you must be willing to rewrite it to .NET :). Yeah, now you know the catch. But either way this is not as hard as it seems and can prove really useful. You are probably also curious how efficient the native VBA code is vs. called C# dll methods. Below you will find an answer to both questions – how to introduce multithreading to your VBA code and how efficient will multithreaded C# methods be against single threaded VBA algorithms.

Multithreading VBA: Sorting an array of random values using Quicksort

Quicksort is one of the most popular sorting algorithms. It uses the “Divide and Conquer” strategy in order to sort a array of values, partitioning the array of value for sorting and applying the QuickSort algoirhtm to each partition recursively. Quicksort is therefore a natural and simple example for multithreading.

Below you can find a simple example of executing a native VBA QuickSort procedure and both a C# QuickSort sequential and parallel method.

Executing C# QuickSort in VBA

Declare Function CreateTestClass Lib "C:\[PATH TO DLL]\TestLib.dll" () As Object
Sub TestQuickSort()
  '---Variables
  Dim i As Long, ar() As Long, testClass As Object, StartTime as Date, EndTime As Date, stp As Long
  Set testClass = CreateTestClass() 'Load DLL for the first time to reduce 1st execution overhead

  ReDim ar(0 To 100000)
  '---VBA sort---
  Call Randomize(100)
  Call RandArray(ar)
  StartTime = Timer
  Call QuickSort(ar)
  EndTime = Timer
  qTime = Format((EndTime - StartTime), "0.00")
  
  '---C# sequential sort---
  Call Randomize(100)
  Call RandArray(ar)
  StartTime = Timer
  Set testClass = CreateTestClass()
  Call testClass.QuickSortSequential(ar)
  EndTime = Timer
  qTimeC = Format((EndTime - StartTime), "0.00")
  
  
  '---C# parallel sort---
  Call Randomize(100)
  Call RandArray(ar)
  StartTime = Timer
  Set testClass = CreateTestClass()
  Call testClass.QuickSortParallel(ar)
  EndTime = Timer
  pqTimeC = Format((EndTime - StartTime), "0.00")

  '---Print---
  Call MsgBox("VBA time: " & qTime & ", C# sequential time: " & qTimeC & ", C# parallel time: " & pqTimeC)
End Sub

Simple right? If you read my post on this is probably nothing new to you. However, as you can see the only setback is really the requirement to reference the C# dll. You can of course add it to your libraries and reference it in other VBA projects using “Tools->References”.

How are the C# methods implemented?

C# TestClass implementation (Unmanaged Exports)

using System.Collections.Generic;
using System.Linq;
using System.Runtime.InteropServices;
using System.Text;
using System.Threading;
using System.Threading.Tasks;

namespace TestLib
{
    [ComVisible(true), ClassInterface(ClassInterfaceType.AutoDual)]
    public class TestClass
    {
        
        [return: MarshalAs(UnmanagedType.SafeArray)]
        public int[] QuickSortParallel([MarshalAs(UnmanagedType.SafeArray)]ref int[] ar)
        {
            ParallelSort.QuicksortParallel<int>(ar);
            return ar;
        }

        [return: MarshalAs(UnmanagedType.SafeArray)]
        public int[] QuickSortSequential([MarshalAs(UnmanagedType.SafeArray)]ref int[] ar)
        {
            ParallelSort.QuicksortSequential<int>(ar);
            return ar;
        }
    }

    static class UnmanagedExports
    {
        [DllExport]
        [return: MarshalAs(UnmanagedType.IDispatch)]
        static Object CreateTestClass()
        {
            return new TestClass();
        }
    }
}

To learn how to build C# dlls which can be referenced from VBA projects see my blog post here: EXCEL: Using C# in Excel VBA

Below you can find also the popular C# QuickSort implementation (from Google Summer of code 2008):

C# QuickSort implementation

public class ParallelSort 
    { 
        #region Public Static Methods 

        public static void QuicksortSequential<T>(T [] arr) where T : IComparable<T> 
        { 
            QuicksortSequential(arr, 0, arr.Length - 1); 
        } 

        public static void QuicksortParallel<T>(T[] arr) where T : IComparable<T> 
        { 
            QuicksortParallel(arr, 0, arr.Length - 1); 
        } 

        #endregion 

        #region Private Static Methods 

        private static void QuicksortSequential<T>(T[] arr, int left, int right)  
            where T : IComparable<T> 
        { 
            if (right > left) 
            { 
                int pivot = Partition(arr, left, right); 
                QuicksortSequential(arr, left, pivot - 1); 
                QuicksortSequential(arr, pivot + 1, right); 
            } 
        } 

        private static void QuicksortParallel<T>(T[] arr, int left, int right)  
            where T : IComparable<T> 
        { 
            const int SEQUENTIAL_THRESHOLD = 2048; 
            if (right > left) 
            { 
                if (right - left < SEQUENTIAL_THRESHOLD) 
                { 
                    QuicksortSequential(arr, left, right); 
                } 
                else 
                { 
                    int pivot = Partition(arr, left, right); 
                    Parallel.Invoke(new Action[] { delegate {QuicksortParallel(arr, left, pivot - 1);}, 
                                                   delegate {QuicksortParallel(arr, pivot + 1, right);} 
                    }); 
                } 
            } 
        } 

        private static void Swap<T>(T[] arr, int i, int j) 
        { 
            T tmp = arr[i]; 
            arr[i] = arr[j]; 
            arr[j] = tmp; 
        } 

        private static int Partition<T>(T[] arr, int low, int high)  
            where T : IComparable<T> 
        { 
            int pivotPos = (high + low) / 2; 
            T pivot = arr[pivotPos]; 
            Swap(arr, low, pivotPos); 

            int left = low; 
            for (int i = low + 1; i <= high; i++) 
            { 
                if (arr[i].CompareTo(pivot) < 0) 
                { 
                    left++; 
                    Swap(arr, i, left); 
                } 
            } 

            Swap(arr, low, left); 
            return left; 
        } 

        #endregion 
    }

As you will notice the above parallel QuickSort implementation utilizes the C# Parallel.Invoke method which allows you to easily leverage multithreading in C# without the hassle of having to manage thread joining etc. The Parallel C# class facilitates the following methods which make multithreading fairly easy:

  • Parallel.For – for iterating over integers
  • Parallel.ForEach – for iterating over any array of objects
  • Parallel.Invoke – basically invokes any number of actions (threads)

Using the For and ForEach linq methods are very straightforward making it very easy to mulithread exisiting singlethreaded algorithms. See a simple Parallel for example for building a multiplication table below:

Parallel.For(0, maxJ, i =>
        {
            for (int j = 0; j < maxJ; j++)
            {
                result[i, j] = i*j;
            }
        }); 

If you would like to see the VBA implementation of QuickSort see below. This implementation was based on this algorithm: Link

VBA QuickSort implementation

Private Sub QuickSort(ByRef values As Variant, Optional ByVal Left As Long, Optional ByVal Right As Long)
  Dim i As Long
  Dim j As Long
  Dim K As Long
  Dim Item1 As Variant
  Dim Item2 As Variant

  On Error GoTo Catch
  If IsMissing(Left) Or Left = 0 Then Left = LBound(values)
  If IsMissing(Right) Or Right = 0 Then Right = UBound(values)
  i = Left
  j = Right

  Item1 = values((Left + Right)  2)
  Do While i < j
    Do While values(i) < Item1 And i < Right
      i = i + 1
    Loop
    Do While values(j) > Item1 And j > Left
      j = j - 1
    Loop
    If i < j Then
      Call Swap(values, i, j)
    End If
    If i <= j Then
      i = i + 1
      j = j - 1
    End If
  Loop
  If j > Left Then Call QuickSort(values, Left, j)
  If i < Right Then Call QuickSort(values, i, Right)
    Exit Sub
Catch:
  MsgBox Err.Description, vbCritical
End Sub
Private Sub Swap(ByRef values As Variant, ByVal i As Long, ByVal j As Long)
  Dim Temp1 As Double
  Dim Temp2 As Double
  Temp1 = values(i)
  values(i) = values(j)
  values(j) = Temp1
End Sub

QuickSort Performance: Native VBA vs. C# sequential vs. C# multithreading

Now let’s analyze the performance of the various algorithms. The benefits of multithreading will be clearly visible when we’ll compare arrays of several sizes and sort them using the various algorithms:

Disclaimer: The following performance metrics are not precise as they simply measure the time that elapsed during code execution and do not reflect the actual processor time. What is more the QuickSort implementations are also slightly different which might also somewhat influence performance. The statistics can nevertheless be used to show the relative performance between VBA and C# (referenced from within VBA projects).

QuickSort: Native VBA vs. C# sequential vs. C# multithreading
QuickSort: Native VBA vs. C# sequential vs. C# multithreading

While it takes almost 15 seconds to sort 2 mln elements with the VBA singlethreaded QuickSort, the C# singlethreaded algorithm requires only slightly above 1 second and the multithreaded algorithm required approx. half a second! For me it was a real surprise that even without multithreading the C# algorithm was sooo much faster than the native VBA algorithm (over 10-12x). In case of the multithreaded algorithm there are of course several performance parameters to consider like the “SEQUENTIAL_THRESHOLD” parameter.

Conclusions: Multithreading in VBA possible, but not in native VBA

As you can see multithreading in VBA is possible if you export your VBA algorithms to C# and reference them from inside your VBA projects. As shown above you can expect significantly improved performance even in singlethreaded algorithms and leverage additional performance boosts thanks to multithreading!. I know that this may seem like a little unsatisfactory – as it’s not actually native VBA multithreading. On the other hand Excel is not a tool designed and appropriate for implementing processor intensive algorithms. What is more, even if VBA multithreading were possible, I hope you noticed that either way even the singlethreaded imported C# method was still much faster (10-12x) than the one in VBA.

You may rarely experience such situations in which your VBA algorithms will take more than a couple of seconds to execute. However, for those of you working in analytics – hopefully this will be a useful tool.

Let me know what you think!

Be sure to checkout a comparison of all available approaches to VBA multithreading:
EXCEL: Multithreading VBA – VBA vs. VBscript vs. C#.NET.

Here you can download the Excel xlsm file and the C# Project: