Hello,
I'm currently playing around with RL, trying to learn as I code. To learn it, I like to do small projects and in this case, I'm trying to create a custom SNAKE environment (the game where you are a snake and must eat an apple).
I solved the env using the very basic implementation of DQN. And now I switched to stable baseline 3, to try out a library for RL.
The problem is, the agent won't learn a thing. I left it to train through the whole night and in previous iterations it at least learned to avoid the walls. But currently, all it does is go straight forward and kill itself.
I am using the basic DQN from Stable Baseline 3 (default values during training. Training happened for 1'200'000 total steps).
Here is how the observation is structured. All the values are booleans:
```python
return np.array( [ # Directions *direction_onehot, # Food food_left, food_up, food_right, food_down, # Danger wall_left or body_collision_left, wall_up or body_collision_up, wall_right or body_collision_right, wall_down or body_collision_down, ], dtype=np.int8, )
```
Here is how the rewards are structured:
```python
self.reward_values: dict[RewardEvent, int] = { RewardEvent.FOOD_EATEN: 100, RewardEvent.WALL_COLLISION: -300, RewardEvent.BODY_COLLISION: -300, RewardEvent.SNAKE_MOVED: 0, RewardEvent.MOVE_AWAY_FROM_FOOD: 1, RewardEvent.MOVE_TOWARDS_FOOD: 1, } ```
(The snake gets a +1 not matter where it moves. I just want it to know that "living is good"). Later, i will change it to have "toward food - good", "away from food - bad". But I can't even get to the point where the snake wants to live.
Here is the full code - https://we.tl/t-9TvbV5dHop (sorry if the imports don't work correctly, I have the full file in my project folder where import paths are a little bit more nested)
submitted by /u/PopayMcGuffin
[link] [comments]
