Hello again!
It's been a while, but after taking a break, (or two, or seven), I finally got around to working on the battle AI based on my machine learning models I've been alluding to. And today I'm happy to say I've gotten to a point where I can share with y'all.
What is Future Sight AI:
Future Sight AI is a computer program that chooses it's next move by looking at the possible outcomes several turns after choosing those moves and selecting the one that leads to the highest chance to win. Using the machine learning models I made for the Pokemon Battle Predictor, the AI can determine how likely it is to win at the end of a given turn and use that information to rank the paths a move choice can take it down. This method, looking at turns in advance and choosing the ones most favorable to you, is the same one used by the best chess playing AIs, and now those same techniques can be applied to Pokemon. However, unlike chess, Pokemon is a game that involves random chance, information unavailable to both players, and a highly variable set "pieces" (aka Pokemon) and their potential moves. These additional factors makes creating an AI for this game in this way significantly more difficult to complete and interesting to solve. Despite those challenges, the result of the application in its current form is an AI that matches the play style of a human player, can take any team and understand how the members work together, and can beat the average player (more on that last part latter).
See it in Action:
If you want to see what the AI can do, I've set up a "live stream" on my website pokemonbattlepredictor.com/future-sight-ai to see both it battling on the ladder along with a log of it's inputs and outputs (you may need to refresh the page a few times to get it going). It will be running for the next day as of posting this. You may notice I've decided to run it on last gen's OU ladder and there's two reasons for that. First, I wanted to test it across a meta that's stable so I know my differing results from day to day are because the AI code has changed rather than Pokemon being added or removed from a tier. And second, the machine learning models it uses are made for specific metas and I wanted to run the AI on the meta it was best at predicting (outside of gen 8). The team it's battling with is nothing special and was made to see if it could understand how different Pokemon can work together (Pelipper activating rain for Swampert, Koko volt switching into Hawlucha for the speed boost, etc), so if you want to see it battle with a different team, send it over (especially since I'm terrible at team building).
How does it work:
There are three main steps for getting an AI like this to work:
The second step is the easiest, and frankly the one I was the most lazy about. This amounts to figuring out what moves, stats, abilities, items, etc. your opponent's Pokemon have so you can make your prediction accordingly. The way it works now is simply taking the moves, items, and abilities, you know they have and matching them to the most common set that matches those known attributes best. I have everything planned out for guessing stats by looking at damage dealt or received, making a machine learning to guess their moves and items based on their team composition, and few other techniques to make it more accurate, but as it is now is good enough and I had to stop working on it at some point ¯\_(ツ)_/¯.
The third step is the one where those aforementioned chess algorithms come into play. Since the AI has a list of its and its opponents next moves, it can run various turns to see what would happen after all the combinations of moves are ran. The way this is done is by using a slightly modified version of the code that runs Pokemon Showdown to make sure the results are identical to what would happen in the actual game. The reason the code is slight modified is to deal with the random chance as instead of choosing an outcome randomly, it will choose one outcome for that battle while creating a duplicate version of the battle that chooses the other outcome so all possibilities can be considered. Once it runs through one turn, it will find its new possible move options for that next turn, predict it's opponents move options, and run through that next turn. As you can imagine, this can quickly cascade into exploring tens of thousands of battle after only looking a few turns deep, so doing all of this quickly is key. That happens until the 15 second timer runs out, which at that point it collects the chance to win of all the different battles that stemmed from a move option and chooses the one with the collective highest chance to win.
How well does it work:
I've tested it a fair bit on the ladder over the past few weeks using random usernames to start at 1000 each time and I've seen it reach up to the 1300s before stopping it. Those results are with it running on my laptop using the CPU to run the machine learning models. It takes ~1 millisecond to run all the calculations for a turn where ~.7 of those milliseconds are spent running the models, so those models are the speed bottleneck. This leads to it being able to look at most 3 turns deep in the 15 second time limit. That's probably as high as it can go given it's current setup, but, the current setup is very limiting. The program is multi-threaded and uses up all the resources on my computer easily, so just running it on a computer that has more cores and power than my laptop would do wonders. Also, running the models on a GPU rather than CPU can yield a ton of benefits speed wise. And, if I was really feeling ambitious, rewriting the code so in wasn't all in JavaScript to be in C++ or something would allow for so much better optimization (have you ever seen how wack multi-threading is in JS). All of this is a long way of saying it may work fine now, but it has some serious potential to get better.
What comes next?
Honestly, outside of making slight improvements or implementing things I decided to table for later, I don't know. With an approach like this, there's really no upper limit to how well it can play outside of how you use the outputs of the highest chance to win models. I will continue experimenting on that front and adjusting the various shortcuts in play to make it run faster, and exploring those ideas with people with people who know the area can help. I'd love to get people's feedback on what they think of it's play or try it with other teams, but for now what I really want is to see what this can do at its current full potential. Oh, and I guess getting it work with double battle too; just imagine what could happen if this could take on VGC...
It's been a while, but after taking a break, (or two, or seven), I finally got around to working on the battle AI based on my machine learning models I've been alluding to. And today I'm happy to say I've gotten to a point where I can share with y'all.
What is Future Sight AI:
Future Sight AI is a computer program that chooses it's next move by looking at the possible outcomes several turns after choosing those moves and selecting the one that leads to the highest chance to win. Using the machine learning models I made for the Pokemon Battle Predictor, the AI can determine how likely it is to win at the end of a given turn and use that information to rank the paths a move choice can take it down. This method, looking at turns in advance and choosing the ones most favorable to you, is the same one used by the best chess playing AIs, and now those same techniques can be applied to Pokemon. However, unlike chess, Pokemon is a game that involves random chance, information unavailable to both players, and a highly variable set "pieces" (aka Pokemon) and their potential moves. These additional factors makes creating an AI for this game in this way significantly more difficult to complete and interesting to solve. Despite those challenges, the result of the application in its current form is an AI that matches the play style of a human player, can take any team and understand how the members work together, and can beat the average player (more on that last part latter).
See it in Action:
If you want to see what the AI can do, I've set up a "live stream" on my website pokemonbattlepredictor.com/future-sight-ai to see both it battling on the ladder along with a log of it's inputs and outputs (you may need to refresh the page a few times to get it going). It will be running for the next day as of posting this. You may notice I've decided to run it on last gen's OU ladder and there's two reasons for that. First, I wanted to test it across a meta that's stable so I know my differing results from day to day are because the AI code has changed rather than Pokemon being added or removed from a tier. And second, the machine learning models it uses are made for specific metas and I wanted to run the AI on the meta it was best at predicting (outside of gen 8). The team it's battling with is nothing special and was made to see if it could understand how different Pokemon can work together (Pelipper activating rain for Swampert, Koko volt switching into Hawlucha for the speed boost, etc), so if you want to see it battle with a different team, send it over (especially since I'm terrible at team building).
How does it work:
There are three main steps for getting an AI like this to work:
- Predicting an opponents move and understanding your chance to win
- Guess what you don't know about the battle
- Looking several turns beyond the current turn to see how the battle can play out
The second step is the easiest, and frankly the one I was the most lazy about. This amounts to figuring out what moves, stats, abilities, items, etc. your opponent's Pokemon have so you can make your prediction accordingly. The way it works now is simply taking the moves, items, and abilities, you know they have and matching them to the most common set that matches those known attributes best. I have everything planned out for guessing stats by looking at damage dealt or received, making a machine learning to guess their moves and items based on their team composition, and few other techniques to make it more accurate, but as it is now is good enough and I had to stop working on it at some point ¯\_(ツ)_/¯.
The third step is the one where those aforementioned chess algorithms come into play. Since the AI has a list of its and its opponents next moves, it can run various turns to see what would happen after all the combinations of moves are ran. The way this is done is by using a slightly modified version of the code that runs Pokemon Showdown to make sure the results are identical to what would happen in the actual game. The reason the code is slight modified is to deal with the random chance as instead of choosing an outcome randomly, it will choose one outcome for that battle while creating a duplicate version of the battle that chooses the other outcome so all possibilities can be considered. Once it runs through one turn, it will find its new possible move options for that next turn, predict it's opponents move options, and run through that next turn. As you can imagine, this can quickly cascade into exploring tens of thousands of battle after only looking a few turns deep, so doing all of this quickly is key. That happens until the 15 second timer runs out, which at that point it collects the chance to win of all the different battles that stemmed from a move option and chooses the one with the collective highest chance to win.
How well does it work:
I've tested it a fair bit on the ladder over the past few weeks using random usernames to start at 1000 each time and I've seen it reach up to the 1300s before stopping it. Those results are with it running on my laptop using the CPU to run the machine learning models. It takes ~1 millisecond to run all the calculations for a turn where ~.7 of those milliseconds are spent running the models, so those models are the speed bottleneck. This leads to it being able to look at most 3 turns deep in the 15 second time limit. That's probably as high as it can go given it's current setup, but, the current setup is very limiting. The program is multi-threaded and uses up all the resources on my computer easily, so just running it on a computer that has more cores and power than my laptop would do wonders. Also, running the models on a GPU rather than CPU can yield a ton of benefits speed wise. And, if I was really feeling ambitious, rewriting the code so in wasn't all in JavaScript to be in C++ or something would allow for so much better optimization (have you ever seen how wack multi-threading is in JS). All of this is a long way of saying it may work fine now, but it has some serious potential to get better.
What comes next?
Honestly, outside of making slight improvements or implementing things I decided to table for later, I don't know. With an approach like this, there's really no upper limit to how well it can play outside of how you use the outputs of the highest chance to win models. I will continue experimenting on that front and adjusting the various shortcuts in play to make it run faster, and exploring those ideas with people with people who know the area can help. I'd love to get people's feedback on what they think of it's play or try it with other teams, but for now what I really want is to see what this can do at its current full potential. Oh, and I guess getting it work with double battle too; just imagine what could happen if this could take on VGC...