THE BELLMAN EQUATION
here are a few values that are going to be important trough this course:-
- s- state of the AI
- a-action was taken by AI
- R-the reward of the action
- y(gamma)-Discount factor
This concept was bought forward by Richard Ernest Bellman
Lets us consider a maze our bot is in this maze here the finish stands for +1 and the fire or end stands for -1 as a result.
- s- state of the AI
- a-action was taken by AI
- R-the reward of the action
- y(gamma)-Discount factor
Let's say,
Our bot begins from start, then it goes left and realizes that it can't go left. then takes the action "up"
then it goes up then takes the next action and keeps going on so on then comes up with the following path by valuing the path's each box as 1.
This happens as the algorithm analyses the boxes before it went to the finish then value it 1 as that box leads it to the reward of +1
now, lets us consider our bot starts from the top-left corner of the maze then the values or messed up.
Hence this is not right
NOW, WHAT??
That is how we use the Bellman equation
THIS IS IT :- V(s) = max(R(s,a)+yV(S'))
V(s) stands for the value of the state
the rest stands for the maximum result of the action + the next state
and as you go near the value increases. Now let us say the AI starts in the bottom right corner the bot knows not to go up as the value is -1 and in left the value is +0.81 and hence goes left and so on to reach the best possible result +1.
Hope the knowledge will help,
to be notified-
Car Enthusiast Check this Page
Thank you
More Knowledge coming every alternative day!
0 Comments