14-06-2002 01:04									#1
	Yet another AI thread Full PM-box? Change here!

Okay here is the new AI thread.

One topic of discussion is NN. I'm still not convinced that this is a bad approach. I'm just not sure that it will be useful in a game where the end game is yet unknown.

I also do not favor scripting languages.

I prefer a modular design to the AI. Each module has its functionality and communicates with other modules (note that this is very NN like). In this manner, a change to a module does not affect the way another module works, only how it reacts to the changed modules input.

Damn, I hate it when I get interrupted. Totally lost my train of thought.

	14-06-2002 02:13									#2
	Support Apolyton buy from Amazon

quote: Is it that hard to think about approaches to tasks without trying them? Why not do a little more research on NN work in practice?

Because I have done some research. But building a model to estimate the theoretical performance of the thing is just not worth the investment in time, relative to actually programming the thing. You know, NN performance isn't that easy to predict, particularly since I really don't know what the functions I'm approximating actually look like.

quote: At least talk to other people who've tried this approach for a game AI and find out what their expereince was. Is that so hard to do?

Well, seeing as how I don't know any AI's in 4X games implemented by neural nets, just a little.

quote: Oh, but it is. While I'm sure doing a basic CS-cookbook NN net isn't especially time-consuming, actually getting it to do AI worth the paper it's printed out on is. This goes for all AI approaches.

Certainly, but it's not such a huge investment in time that if you use the wrong approach initially, it could destroy the project, or that a couple approaches can't be worked on at the same time. Hell, I was always under the impression that most strategy games go through multiple incarnations of the AI.

	14-06-2002 02:20									#3
	Support Apolyton, buy Civilization 2

Hey Ramo,

I've looked over your NN code that you posted and I'm a little perplexed. The other code example that I saw used one nueron as the base class, then there was one layer class, lastly a net class. This seemed a little simpler to keep track of then your version. What'cha think?

R:PM

	14-06-2002 02:47									#4
	Support Apolyton, buy Civilization: The Boardgame

I know it was a little messy. There are a bunch of aesthetic changes I would make if I did it over again (what I had in the end didn't turn out all that much like what I started out with). But I don't know if it warrants a code rewrite since users of a neural net only need to know its interface, which I hope was clear enough, and I don't think there are any major design flaws in it.

	14-06-2002 03:23									#5
	Lose 30 kilos (of popups)

I'm guessing that it wasn't clear enough for me, although java is still a mystery to me (I think I know every other lang but that one though). I'll give it another shot.

	14-06-2002 03:27									#6
	Browse Apolyton AD-FREE

Anything in particular needs clarification?

	14-06-2002 03:45									#7
	Inflate your Upload Space

Ramo, you could talk to a coder who doesn't use NN, but is familiar with him. I bet he would have some insights. Even if you decide to continue down this path, some insights about what the hurdles are would help.

Another option is to look for where NN have been used in something analagous. Moomer mentioned a game that used NN. You could check that out.

	14-06-2002 04:25									#8
	Got spare money?

No, GP. I mentioned BC3K as an discouragement. It's quite possibly the worst disaster to ever hit the unsuspecting gaming audience. No, let me retract that; it's up and away the worst gaming disaster ever - leaving the rest of the shoddy but numerous field competing for that sad title in the dust. NN was but one of the many - many! - things that went wrong with it. But I sure suspect it contributed heavily.

Not that there isn't a lesson to be learned from BC3K. But I suspect it's more for Vel than for anybody looking for a sane implementational role model. Heck, it was so godawfully bad it doesn't even work as a "don't go there", because no one would actually do that.

It's quite beside the point really. If all the programmers on the CB team were blind, mentally disabled and constantly high on any given assortment of drugs, be they legal or not, they still wouldn't come close to coming up with a BC3K equivalent.

But I still would like to hear why NN is such a hot track - that entire technology is used in fields quite removed from gaming AIs, and the problem areas that it handles well do not overlap game goal resolution, as far as I can see. So I'm curious - it'd be very nice if they could actually come up with a worthwhile gaming NN application, but I sure would like to see some reasoning why they think its even possible.

	14-06-2002 05:58									#9
	Support Apolyton

quote: and the problem areas that it handles well do not overlap game goal resolution

Why do you say that?
Optimizing the game resolution is nothing more than optimizing a highly specialized variant of a "score" concept.
There are a series of inputs and essentially an output (actually, there is a series of outputs, and some linear combination of which makes the "score").

And of course, neural nets are nothing more than varying function approximators.

	16-06-2002 18:45									#10
	Support Apolyton, buy Galactic Civilizations: Deluxe Edition

quote: Originally posted by Ramo Why do you say that?

Because it's the Conventional Wisdom on this subject. Doesn't have to make it the Whole Truth, true, but I strongly feel the burden of proof is with those who challenge it.

NNs do well in resolving familiar patterns from noise. This make them a useful tool in tasks such as image interpretation, industrial recognition filtering and near-realtime spacial resolution problems.

However, all these problems have two traits in common: a) they are mostly analysis of a state, rather than analysis of a stream and b) they rely rather heavily on there being actual qualifiable entities to resolve - that's to say the NN "knows" that it's looking for, say, a ball (a sphere) even if part of it is occluded by a bush.

Now, defining geometrical primaries is trivial, and most real-world targetting is not to difficult either - we know how the tumor we want to look for looks like on the brain scan. But mapping abstract concepts - such as game goals - into reasonable target states is amazingly difficult, and has failed miserably even with such well defined games as chess, even though there is solid evidence in place that the human brain actually does that.

So while I think your drive for using NNs is interesting I really would like to know how you intend to actually use NNs to first identify gamestate and then derive courses of actions based on that identification.

	16-06-2002 19:07									#11
	Enter the AD-FREE zone

1. It's just some cool thing he learned in school...that he doesn't really know about from doing it.

2. He doesn't feel any reason to want to learn what types of programs work in practice for what types of applications. he feels no urge to learn about practical lessons learned the hard way. He thinks the only way to test a project approach is to do it. (i.e. it is impossible to learn from experience.)

3. And he feels no impetus to deliver for the team. He has absolutely no problem with failing to deliver.

(My words are provacatively stated...but meant.)

	16-06-2002 21:37									#12
	Avatar Enlargement: We've got the solution

Just a couple of points and observations.

The team doing the AI research has my complete support, backing, and faith. I am quite certain that they will explore all the possibilities, and come up with some manner of implementation (NN, or some hybridized approach) that gets the job done.

What we're designing is a rules based system. Maybe we'll discover that NN's have limitations that make working with a rules based system inadequate for our purposes, or perhaps, we'll discover that we can use them at the root level, and have something very much akin to an "expert system" in charge of making military decisions and such from a tree-based structure.

In truth, I don't know enough about that particular end of it to really know WHAT the end result will look like.

I do think, however, it should be possible to quantify what, precisely, constitutes good play in terms of the game itself (i.e. - Attack, Y/N? If AI total Firepower is equal to or greater than 125% of defender total Firepower, then yes, else build more troops...that sort of thing).

As to failing to deliver for the team. I always like to refer back to baseball when thinking about stuff like that.

The very best baseball players in the leage have....what? a .300-.350 batting average? Which means that they hit the ball 30% of the time. Or, looking at it another way....they fail to deliver for the team a whopping 70% of the time.

And these are the best of the best.

Success is built on the framework of failure. Not failure in the sense of giving up, to be sure, but failure born from trying something, having it blow up in your face, and then trying something else till you get it right.

I have no problem supporting an effort that ultimately fails. Baseball (and most everything else we humans do) shows us that we will fail far, far more often than we succeed.

But if we keep at it....if we don't give up the search....we'll get there.

Like Ramo, I do my best learning by doing. I read all about fencing, and thought I understood how to do it...and in fact, I did know a fair bit about it....there was some value to be had in reading and studying about it.

Nonetheless, it wasn't until I actually picked up a foil and had a go at it that I truly understood the nature of the beast.

What I discovered was that reading about fencing....studying and thinking about fencing was a far, far different creature than actually fencing. I learned more in my first (90 second) duel (which I lost...HORRIBLY, I might add) than I learned in the entire time I spent studying about it.

It's stuff like that, that serves as my point of reference, and why I'm not afraid of failing.

What I am more afraid of is giving up, or not trying a thing at all.

If you never try....if you never open yourself up to the possibility of failure, it's true....you will not fail very often.

It's also true that you won't succeed very often.

Success is in the trying.

-=Vel=-
(just some random thoughts on a Sunday morning)

	16-06-2002 23:11									#13
	Support Apolyton, pre-order Civilization IV

Thanks, Vel. I completely agree.

quote: So while I think your drive for using NNs is interesting I really would like to know how you intend to actually use NNs to first identify gamestate and then derive courses of actions based on that identification.

I've been meaning to post something like this for a while.

Essentially, the design is bottom-up. Based on the parameters given from a "Commander," (as well as the relevant data it needs) a game object makes its decision. Each sort of game object (armies, buildings, and so forth) has its own NN, as well as certain "helper" NN's, such as army firepower (not entirely sold on that idea yet, though).

For this example, I'll describe an army NN.

It has a series of "fixed inputs." These inputs aren't modifiable by the NN. First is the army firepower. Then there are a series of "province information inputs"; each province has some data associated with it - the firepower of enemies, friendlies, and neutrals, and fortifications and the ownership of the province (the NN province data model likely will be streamlined based on performance constraints). Then there is a "variable input" - which province to move to. All of these are, of course, easily quantifiable.

As for the outputs, first there are the "score outputs." These include the change in military, domestic, diplomatic, and economic power within a certain radius proportional to the firepower of the unit since the command was issued. There are also "commander outputs," such as whether a certain province is under player control (probably will be changed, but that's all my Commander model has for the moment).

My main worry is performance, which again, is completely dependent upon what the functions look like; something I do not know.

quote: 2. He doesn't feel any reason to want to learn what types of programs work in practice for what types of applications. he feels no urge to learn about practical lessons learned the hard way. He thinks the only way to test a project approach is to do it. (i.e. it is impossible to learn from experience.)

Look, there isn't going to be any academic paper about a good input/output model for a grand strategy game, and certainly not what functions based on various models would look like. This is just not the sort of thing that I could pick up a book and read about. I really have to actually do it.

quote: 3. And he feels no impetus to deliver for the team. He has absolutely no problem with failing to deliver.

If I fail to deliver with the NN (which I seriously doubt - I know there are some uses where it will be useful to at least some degree), the more conventional approaches will get the job done.

	17-06-2002 20:07									#14
	Browse Apolyton AD-FREE

quote: Originally posted by Velociryx What we're designing is a rules based system. Maybe we'll discover that NN's have limitations that make working with a rules based system inadequate for our purposes, or perhaps, we'll discover that we can use them at the root level, and have something very much akin to an "expert system" in charge of making military decisions and such from a tree-based structure. In truth, I don't know enough about that particular end of it to really know WHAT the end result will look like. I do think, however, it should be possible to quantify what, precisely, constitutes good play in terms of the game itself (i.e. - Attack, Y/N? If AI total Firepower is equal to or greater than 125% of defender total Firepower, then yes, else build more troops...that sort of thing).

Be aware that expert systems (whether rule-based or fuzzy) have very little to do with NNs. In fact, the two approches to AI (or just semi-intelligent filtering, in most cases) are at ends with each other. Lining up a solution to a problem that facilitates one of these approaches will necessarily make the other approach more difficult.

Unless some reasoning emerges as to why you think it's a good idea to mix expert-system style rule-based decision making with NNs I'd say you'll better prepare to exercise a hefty dose of that faith you mentioned, Vel.

	17-06-2002 20:18									#15
	Support Apolyton buy from Amazon

quote: Originally posted by Ramo For this example, I'll describe an army NN. It has a series of "fixed inputs." These inputs aren't modifiable by the NN. First is the army firepower. Then there are a series of "province information inputs"; each province has some data associated with it - the firepower of enemies, friendlies, and neutrals, and fortifications and the ownership of the province (the NN province data model likely will be streamlined based on performance constraints). Then there is a "variable input" - which province to move to. All of these are, of course, easily quantifiable.

Unless you're per-object NNs are really simple - how many simulated nodes do you calculate you need for you're particular matching needs? - it sure seems to me as if you're setting yourself up for some serious number-crunching with this approach.

Also, if the senario you described above is typical for the evaluation your AIs will have to do then there's absolutely no need for NNs - if we are talking about simple comparison and evaluation of raw data - as opposed to aggregated data states, which you don't seem to have in the above model - then well established practices such as rule models or goal matrices will do very nice indeed, with a known cost in processing and easily tweakable AI logic - and behaviour.

But I've said about as much as I intend to say. It's your game, and I wish you the best of luck. I'll be very interested to see if your NN approach goes anywhere.

	18-06-2002 02:47									#16
	Support Apolyton, pre-order Civilization IV

quote: Unless you're per-object NNs are really simple - how many simulated nodes do you calculate you need for you're particular matching needs?

About 20 to 30.

quote: it sure seems to me as if you're setting yourself up for some serious number-crunching with this approach.

That may be true. I've got a couple ideas to streamline the approach provided the need arises.

quote: as opposed to aggregated data states, which you don't seem to have in the above model

Why do you say that?

	17-08-2002 07:14									#17
	Put an end to popups!

Sorry to revive a thread and discussion that may not be an issue anymore, but could somebody fill me in on the current state of NNs in CB? Ramo, I see you posted some code somewhere, care to lead me to it? Even if I'm not volunteering to actively start writing code (no time), I have experience with industrial NNs and expert systems, maybe I can be of help. I'm mighty curious how you want to use these to handle both larger, meta game issues and smaller tactical decisions.

Somehow I feel that decision trees are a better learning algorithm than NNs if you want to have in game learning, and I am especially interested in where, and when you're planning or experimenting to do the learning of the NNs. I don't think this is a simple thing to balance, rule based systems have many disadvantages, but at least are easily controllable... the biggest problem I see with NNs is that small changes to the setup of NNs can lead to big behavioral changes, e.g. the difference between using 10 or 11 hidden nodes could mean a world of difference. It would be very cool if you manage to make the first adequate use of NNs in strategy games I'm aware of, but I hope you didn't put the goals too high to achieve.

I'm really very interested in this, I hope you don't mind letting me peek into your code. And don't worry about messy things, you should see my code when I'm trying out things

DeepO

	27-08-2002 01:07									#18
	Lose 30 kilos (of popups)

DeepO, I posted a thread on AI in the "Candleworks" forum which had the code, but I can't seem to find it. In fact, a lot of older threads seem to be gone... I'll see if I can find it on my hard drive tonight.

More info on the state of NN's in CB are in the thread you posted.

	27-08-2002 15:05									#19
	Support Apolyton, buy Civilization 2

Ramo, indeed, let's take this discussion in house... I made the previous post before I made the CB team... I'll answer the posts you made in that other thread later today

I'm still interested in the code, so if you find it, please e-mail it to me.

DeepO

	27-08-2002 22:10									#20
	Get a bigger avatar today!

I'm not a programmer. Am I correct in saying that NN's allow the AI to "learn" as the game progresses?

	27-08-2002 22:19									#21
	Get a bigger avatar today!

yep, that's one of the features of NNs... but there are other techniques that have the same properties. And I believe these are better suited for CB (which is what the discussion revolves around)

DeepO