Scott February 2016

How to cure memory allocation issue on UI thread which is not an issue on command line?

I have written an algorithm (function) to read in a number of files from a list, manipulate and agglomerate those files, and then store as a single file. This works fine and processes each file in under a second and a half.

To improve the user experience I created a simple GUI to prep the algorithm (input file list, output file location, and processing options etc.). This works fine.

However, the algorithm takes significantly longer to execute on UI thread (Windows Forms Application), over a command line thread (Console Application). I can find no explanation as to why. The command line execution time is close to linear with the number of files. However the UI thread or BackgroundWorker thread execution is very non-linear and very quickly becomes too slow to be useful. See table below. (I killed the 100 file run after 2 hours).

Number of files 1, 3, 5, 13, 20
Command Line Execution Time 1s, 4s, 7s, 19s, 29s
UI BackgroundWorker Execution Time 1s, 7s, 22s, 309s, 441s

I thought this was to do with the BackgroundWorker class I was using to perform the processing, however on removing the backgroundWorker and calling the function directly from a button press caused it to be even slower (82 seconds for 5 files (and caused UI to lock)).

I have now created a standalone static test function, scattered with Stopwatches to performance test and discover the issue.

One thing is highlighted as getting slower and slower with every additional file.

stopwatches[21].Start();
byte[] canData = reader.ReadBytes(8);
stopwatches[21].Stop();

"reader" is BinaryReader. This was used as both simple files, and ZipArchiveEntry files are being processed.

Other lines of code which are highlighted as slowing, are always memory allocations, either a "new" statement, or List<>.Add functions.

My understanding of BinaryReaders involves them buffering incoming information and therefore

Answers


Scott February 2016

Fault Finding

After creating a working UI that was just as fast as the command line (new trivial UI), I started looking for reasons that my application was performing poorly, I systematically removed code, call backs, methods, data, eventually deleting all components as well, nothing helped. I now had two identical pieces of code, one working well, one not working... So I systematically did a diff of the files in each project folder, tried to understand the important differences (ignore names changes etc.), so I found what I thought could be the issue. I then altered the file outside of Visual Studio, rebuilt and ran... and performance was good. Success, fault found.

The Answer

The fault was in the csproj file.

<Reference Include="System.Core">
  <RequiredTargetFramework>3.5</RequiredTargetFramework>
</Reference>
<Reference Include="System.Net.Http" />
<Reference Include="System.Xml.Linq">
  <RequiredTargetFramework>3.5</RequiredTargetFramework>
</Reference>
<Reference Include="System.Data.DataSetExtensions">
  <RequiredTargetFramework>3.5</RequiredTargetFramework>
</Reference>

Which I modified to the following

<Reference Include="System.Core" />
<Reference Include="System.Net.Http" />
<Reference Include="System.Xml.Linq" />
<Reference Include="System.Data.DataSetExtensions" />

Note both had

<TargetFrameworkVersion>v4.5</TargetFrameworkVersion>

Also: I still can't find where in Visual Studio where this option is set, I did go through both sets of properties before looking in files to check, but even after finding the issue it's still hidden to me in Visual Studio.

I believe this is because I copied the Form from a Form that was upgraded from Visual Studio 2008, and the upgrade process m

Post Status

Asked in February 2016
Viewed 3,735 times
Voted 7
Answered 1 times

Search




Leave an answer