For developers: Idea: Benchmark visualization program

We have so much things to work off that we cannot do them all

So i wanted to throw my idea out what i think would be cool to have: a little app that starts the benchmark via launching the beamng console and reads the results back over json file output in order to visualize them nicely

So the task: write a nice GUI around that

See attached files (to be unpacked in main game folder or as mod .zip) for proof of concept launch args, and lua benchmark that will output in json files (i added some example output files). (Lua writing from the console is currently broken, will be fixed within next update)

- - - Updated - - -

UPDATE: Idea2: write a .html file as output directly, that would be cool as well

 

Here are some screenshots from an app I made for this. It runs the benchmark, fetches the results and generates a html chart from the data.
Features so far:

• Run Banana Bench
  • Single run
  • Best of five
  • Average of five
• Retrieve JSON data
• Generate an interactive Google Chart
• Realtime threshold indicator
  • Indicates which realtime reading is still playable (about 120- 140% for me, but I guess I'd need more data to make it a bit more accurate)
  • Can be used to easily gauge how many cars you can run at once
• Export chart
  • As image
  • As HTML

Potentially planned for the future is some sort of upload to the BeamNG team for easy spec comparision. I did not include any PC spec listing in the diagram yet, but that is also planned (Not sure yet where to put it).
Let me know what you think

 

Very cool work, well done!

Is it open source?

 

It's not open source currently, but I do plan to make it OSS. It's going to be a git on bit bucket or rather already is, just private.

 

Absolutely.

 

Like so?

 

I also think there should be a clear "100%" line since that is probably the most important thing.

 

Well that's what the "Realtime threshold" thing is about. It currently sits at 140% since I found that everything below ~120% is laggy for me. I am not sure why, but 100% doesn't seem to be the limit on my computer. (6 cars work just fine, 7 are laggy already even though in theory it's above the 100% mark).
I chose 140% for now as that's a nice point inbetween 6 and 7 cars on my machine (and 120% *looks* like 7 cars still work), but I can tweak that value to whatever works best.

 

Is anyone here willing to test my benchmark tool? I'd like to get a few results from different systems to see if everything is working fine before making it available to the broad public.
Just let me know via PM.

 

Just a little update: I am currently working on some sort of upload feature, so users can upload their benchmark results to a central database.
The app will then display some of these results as some kind of a high score list.

I also had the idea to add a feature that rates the user's computer based on the results and tells him how good the computer will be CPU wise.

 

I already have something like this set-up, just not sure yet if it will stay this way.
Currently I basically have a list of 5 different setups, ranging from a "Low Score" to an "Extreme Score", where it displays the graph for each of these. You can toggle what you'd like to display with some expanding area things in the app.

If you got any nice ideas for the visualization, please let me know.

- - - Updated - - -

Here are some current screenshots, they are probably not final yet, but this is the direction it's heading

 

Here's a new thread with some more information:
http://www.beamng.com/threads/13099-BeamNG-BananaBenchmark-The-visual-approach