Hey everyone, thanks for taking the time to read my post! I'm doing research in the accuracy of games their physics engines and I want to test this game as well. Problem is, I have no clue how to log the data to an external file for analysis. I was hoping there would be some kind of way to export to a CSV file, but thus far I have not found helpful posts on the forum. One thread that had what I wanted ended on an unsatisfactory note, leaving me here. I have looked through the wiki, and I did see this post. It looks like what they use to make UI elements, but maybe it can be used to export data? I don't need to log a lot of data. Preferably just time, speed, coordinates, and g-forces. I did see all of these values in the Sensors Streams, but I'm clueless as what to do next. I hope some of you know what I mean and know of a way to fix it. Thanks for reading!
You can convert json to csv: https://json-csv.com/ From UI app list you can find log available stream app which you can use to log json: I guess it records sample each second or so. @Diamondback he mentioned "time, speed, coordinates, and g-forces" in the post.
Thanks for the replies! I appreciate you guys helping me. Anyway, I have looked into the UI app, and it seems like the thing I need. I do believe that the exported file is in "Documents/BeamNG.Drive/loggedStreams.json" and putting it through the converter gave me pretty much all the data, but as far as I could see, only the maximum and minimum values. There are also some ExampleVals, but I'm unsure as to what they do. Graphing the data leaves a lot to be desired. Here are some links to the Replay, the Excel and the CSV file. Please note that for the replay I used an automation car. Link can be found here: https://www.beamng.com/resources/bmw-serie-5-2017.6235/ As for the files, here is a Mediafire folder. The forum would not let me upload my Csv file. http://www.mediafire.com/folder/akrspgl9j7s4s/BeamNG I would love to be able to leave out undesired information, like the lights_state, but it's no problem to filter it out afterwards. The most important thing right now is to log data in time intervals. Something like every few frames, to get some nice and accurate graphs. Would that just require some spamming of the logging button or is there another way?
I think you need @Diamondback for that, he can be busy so you might need to send private message to him. There is no possibility to change logging interval that I know of, current logging options are bit less than in other simulators in that sense, but I do hope they will get better in a future. What I know is that you can use LUA to print values to file for example and that can be run every frame, but to make that happen requires of course bit of coding. User Arcanox has probably skills needed, not sure how much time or interest he has, but asking him is option too. This post however has pretty good information that might help you if all else fails: https://www.beamng.com/threads/lua-help.56996/#post-905294 I'm quite curious of your findings between different games, will there be published article or paper somewhere at sometime?
Supermuskox messaged me about this and I'd be happy to help. I can probably write a script to do this but I may not have time for at least a week, but I can give the general premise. It's probably easiest to do this from a Lua controller rather than a UI app simply because there's less boilerplate required for the controllers. I'm also not positive that the UI apps can write files. I know Angular apps can't natively write files, but there may be a BeamNG API that's provided that I'm not aware of. In the Lua controller, you'd want to open a file such that it gets wiped first, and close/reopen the file when the vehicle is reset (allowing the output stats to be reset). In the update method, you'd use the dt parameter to keep track of how much time has passed since the last log, and if it reaches the threshold to log a new value, do so. It'd be a simple task to write the logs to a CSV since it's a very simple text-based format. Using a controller would also allow keybinds to be set to start/stop/reset the logging, as opposed to a UI app which you would have to click (I don't believe UI apps receive direct keyboard events).
Thanks for posting! I have been trying to find the things I need and I think I've found them. All files mentioned from now are in ..\BeamNG.drive\lua\vehicle. Starting off with sensors.lua. This one contains almost everything needed. Line 12 to 14 define M.gx, M.gy and M.gz. Position is found. Second thing to log, speed. I have not found a variable that defines this, but the function getVelocity() seems to do the trick. Examples in scriptai.lua, ai.lua and bdebug.lua. According to the documentation, updateGFX(dt) should return DeltaTime as an argument, so that might be a way to get time logged. For writing the file, WriteFile() seems like the way to go. Should not be too hard. Problem is, I have no clue how to continue with this. Like, where do I put the Lua file? Or do I edit the existing Lua files? What code initializes the script? If anyone knows how to continue, please let me know. And to answer your question @fufsgfen, it's for a project I'm doing in school called a PWS or "profielwerkstuk". Which is basically a big research project you do at the end of Dutch high school. So, unfortunately no published research, but I can upload some nice graphs from the data I acquired. Deadline for the project is in about two weeks, so I guess you'll get to know pretty soon.
I can write a simple one next week; you'd want to put it in its own mod folder and attach it to a JBeam part. I can give you a place to start off and you can modify it as you see fit
With a lot of help from @Arcanox we created a script that can log data to a .csv file. Put the .zip file in your mods folder (Documents\BeamNG.drive\mods), or unpack it and put it in the unpacked folder (Documents\BeamNG.drive\mods\unpacked) if you want to edit the script. Default vehicle for logging is the ETK S Series, which is a modded vehicle. Be sure to edit the script if you want to log other vehicles and or other types of data. You can enable the script here: Once again huge thanks to Arcanox, for putting it all together. Anyways I have made some graphs from the data I have collected. This is the result from slowing down two similar cars in real life and in BeamNG from 30km/h to 0. Real life: Simulation: Graphs are made in Dutch but I think they are self-explanatory. It's pretty interesting to see that BeamNG makes a little dip at around 2 m/s and I'm not sure what causes it. If anyone has a theory, please let me know, as I'm pretty interested myself.
Dip is probably because of nodes start to create more rolling resistance at that point, tires are not exactly round, but are like pieces of plank put toghether to resemble a wheel, so there is bit higher rolling resistance at very low speeds. At higher speeds that has no effect to rolling resistance anymore. You can also see this when you roll car slowly it kinda shakes, it is also possible to see if you enable debug view. Where most other sims have just one contact point to ground per tire, BeamNG has several, which form a contact patch, how many contact points there is depends from tire pressure, size node count etc. Then some sims have very simple chassis flex simulation, nothing even nearly as complex as BeamNG, something that is quite challenging to analyze, like is acceleration of car body around 3 axis, which is what I think where other sims are way behind. If using modded vehicles to compare game to real world, you might end up with not so good data, for example there has been several changes to game since S series had last update, so I would recommend to use default content in game.
i've been following this thread/conversation recently. i'm a student, and currently doing a simulation research using beamng. i'm not very good with programming unfortunately, it will be so kind if you can provide a video on how to used the script and where to gain the datalog because i'm having a hard time finding the data needed after following the instruction given
Fun to see this post get some attention again! Hope you can collect some useful data. I tried to use the mod again as well, but it looks like something broke in the datalogging.jbeam file, seeing how it does not show up in the additional modifications for me. The problem is, that I have no idea how to fix it. My best suggestion is pinging some folk who actually know what they're doing. If anyone would be able to take a quick look through the .lua files, and see if anything isn't configured correctly, greatly appreciated. Totals out to about 120 lines of code, so it should be very doable.
Hi, I just tested it and it still works accordingly. Maybe you forgot to assign the right vehicle name in the jbeam file ? Code: // Copy the following section and change all "etkss" to the ID of the vehicle that the logger is to be installed on // (e.g. for the bus: citybus_datalogger, citybus_mod, etc) "etkss_datalogger": { "information": { "name": "etkss Data Logger", "authors": "Arcanox" }, "slotType": "etkss_mod", "slots": [ ["type", "default", "description"], ["common_datalogger", "part_datalogger", "Data Logger Logic", { "coreSlot": true }], ] } The instructions clearly say that you need to adjust the vehicle name into which vehicle you want to apply it to, otherwise it won't show up in the additional modification slot. The example uses 'etkss" as a vehicle but that would not apply to vanilla vehicles. Just tested etki and that worked immediately.
Weird, I tried changing etkss to citybus before posting that, but I did not see the datalogger component appear in the additional modifications tab. Just now, I tried to replace etkss to etki, grab a car from the ETK I-Series, but no additional modifications slot. Am I missing something here? This is how that snippet of code looks for me right now: Code: // Copy the following section and change all "etkss" to the ID of the vehicle that the logger is to be installed on // (e.g. for the bus: citybus_datalogger, citybus_mod, etc) "etki_datalogger": { "information": { "name": "etki Data Logger", "authors": "Arcanox" }, "slotType": "etki_mod", "slots": [ ["type", "default", "description"], ["common_datalogger", "part_datalogger", "Data Logger Logic", { "coreSlot": true }], ] } I'm on version 0.20.2.0.10611, if that helps.
No, I don't see something wrong, it has 3 imes the name change and all characters seem into place. Technically it even needs only one change and that is the slottype, but better stick to clarity. Anyway I have no idea what you are missing. I had the mod before but yesterday I especially downloaded the zip file again to make sure. --- Post updated --- Why don't you try mine it's actually the one I used yesterday. So funny I was just going to bin it before I read your post
You have got to be kidding me, your version DOES work for me... I have no clue what is happening here, but something is seriously scuffed in my install. Anyways, thank you for sharing your version of the datalogger. Glad it works again, so I can get back to the reason this thread was revived. So what you want to do, is unpack the mod. I recommend dragging the zipped mod into ~/Documents/BeamNG.drive/mods and then in the game, go to the "Mods" tab, press the datalogger mod, and then "Unpack". This will move the mod to the ./unpacked directory in the ../mods folder. From here, you will have to do a little bit of editing. Go to the ./testdatalogger/vehicles/common directory and open the datalogging.jbeam file in your text editor of choice. Now you want to change all the instances of etkss to the ID of the vehicle you want to datalog with. It seems to me that the vehicle ID is the name they give to the car in the internal files. So if you browse to your install location, and then BeamNG.Drive/content/vehicles/, you should find all vehicles, with their ID being equal to the filename of the .zip. After that, go back to the BeamNG application, and start up a world. Grab the car you just entered in the .jbeam file, and go to Vehicle Config (The gear icon in a box). You should see a box named "Additional components" in that list. Select the datalogger part here, and start driving. If you're done with logging, remove the component, and check the ~/Documents/BeamNG.drive directory. A new .csv file should have been created, with the logged data. If you don't see the additional components box, be sure to check that you didn't forget any commas anywhere, otherwise, you might have the same issue as me? Hope that covers it for you!
Hahaa glad I could help Just two one minor detail to fill in on your explanation : 1. The vehicle names don't always match their zip file names, you should unpack them to make sure you get the correct naming. So when you unpack a vehicle .zip you see a folder called 'vehicles'. Open it and next you see is the correct name of the vehicle. 2. The slot you name 'Additional Components' is actually a slot named 'Additional Modifications'.
IT'S WORKING!! thank you so much to everyone who has contribute in this thread, this mean so much to me. currently my faculty is trying to developed a driving simulator for research purpose, and my research currently is to validate the beamng drive as a simulator software. there's a stigma where games is view as child play and deem to be useless as a research tool, but then again i have reviewed some other professional and valid simulator software. most of them is either highly price or lack of realistic graphic experience, most simulator software also aren't quite user friendly as they have to be setting manually with programming language, most of us who study automotive engineering wasn't expose much to programming language in our course. i'm amazed how beamng developer put their effort in building this game, it is user friendly, crazy graphic and physics, and the amount of user interface that call out the body condition of the vehicle is surreal. i think there's a lot more can be done with this game. especially with a great community like you guys! and again, THANK YOU!
I'm trying to get is working on the Barstow but the Additional Modifications button won't show up, is the mod outdated or is this my PC's problem? I really want to try it!
Read instructions here and also in files inside the zip and you should be able to adapt this to work with Barstow too, by default it works only with etki: https://beamng.com/threads/data-collection-and-logging-to-a-file.58863/#post-1229573 Technically it should be possible to add this to all vehicles, also I think it should be possible to extend what is being datalogged quite a bit as currently it is more of sample how to get started and truly a great one at that.
