AI Driver Assist + TTS Aural Feedback

Discussion in 'Utilities and programming' started by HighBeam9000, Aug 21, 2023.

  1. HighBeam9000

    HighBeam9000
    Expand Collapse

    Joined:
    Aug 2, 2021
    Messages:
    345
    Hello!

    There's some interest in accessibility mods for BeamNG. I recently conversed with another user on the Forum, @TheBlindSaiyan, who has some great ideas to assist visually impaired players such as:
    • NVDA Text-to-Speech (TTS) Support -- This functionality will likely take the form of a plug-in of some sort that exposes some kind of sayThisText() function. I'd like to make such functions available to all modules running in the game to make TTS services available to all.
    • Remap Some AI Controls -- This functionality will probably come in the form of an alternate vehicle controller to be used when AI is driving the vehicle.
      • Some control inputs (throttle, brake, maybe shifting) will be wired to the player's controls instead of the AI. This will allow the AI to steer while the player controls the throttle and brakes.
      • The now-unused AI control outputs can be piped to a TTS module that tells the player about the AI's throttle and braking "suggestions", possibly via events that other modules can register for.
      • This "hybrid" control setup can help players learn the AI-preferred racing lines, and enable visually impaired drivers to enjoy BeamNG more.
    • Terrain and Turn Announcer -- This functionality would require the TTS module. If the player is currently on a road with specific embedded information, such as AI-enabled roads, the announcer would look ahead and call out certain features such as turns, forks, road surface changes, etc.
      • Helps visually impaired drivers enjoy BeamNG.
      • Might be a basis for a rally co-driver for turn callouts & stuff.
    • Menu Announcer -- This announcer would watch for the menu highlight moving around, and would announce the text of the highlighted menu.
      • Helps visually impaired drivers enjoy BeamNG.
      • I don't even know if this is possible, but maybe it is.
    I'm trying to help with some mod work, but I'm not as "expert level" as I'd really like to be. I'd like to know if anybody who is expert level would like to help with this endeavor. Any help would be appreciated. Most of the help needed would be technical expertise because I'll probably be tinkering with technical stuff that I've never tinkered with before.

    Please reply with your questions, comments.

    Thank you for taking the time to read this post.
     
    • Like Like x 3
  2. Happyhusky

    Happyhusky
    Expand Collapse

    Joined:
    Sep 9, 2023
    Messages:
    502
    Why isn't this getting funded??
     
  3. HighBeam9000

    HighBeam9000
    Expand Collapse

    Joined:
    Aug 2, 2021
    Messages:
    345
    I'd love to work this as a job.

    Sorry I haven't posted here in a while. My mod research hasn't yielded much up to this point that would help in this area. Also, I has some family stuff to deal with.

    However, I just learned about the throttle and brake overrides (electrics.values.throttleOverride & electrics.values.brakeOverride). I believe that these signals are what the AI and cruise control use to take control of the throttle and brake functions from the player. I don't think these signals can be used to implement a hybrid player/UI driving model because if we'll be fighting the AI for control of these signals. I still believe that the most straight-forward implementation route would be a custom vehicle controller that routes control signals differently than the normal controller.

    The normal controller routes control signals like this
    player throttle --> throttle
    player brake --> brake
    AI throttle --> throttle override
    AI brake --> brake override

    The custom controller might be rewired like this:
    player throttle --> throttle
    player brake --> brake
    nil --> throttle override
    nil --> brake override
    AI throttle --> custom aux throttle
    AI brake --> custom aux brake

    Setting the override signals to nil makes them have no effect. Need more vehicle controller research...
     
  4. Happyhusky

    Happyhusky
    Expand Collapse

    Joined:
    Sep 9, 2023
    Messages:
    502
    You could check out the Me-262 autopilot mod and see how they implemented it onto an aircraft.
    --- Post updated ---
    They also didn't use the ingame ai I'm pretty sure
     
  1. This site uses cookies to help personalise content, tailor your experience and to keep you logged in if you register.
    By continuing to use this site, you are consenting to our use of cookies.
    Dismiss Notice