Lexer

I’ve just completed a Lexer in order to produce a stream of tokens for the Parser.

There’s some parts of the Dragon Book which I don’t really understand the reasoning. Such as the Lexer adding info to the Symbol Table… why would it do that? It’s the Parsers job to build a syntax-tree, that seems like the correct place to add symbols. The Lexer is complicated enough just producing a stream of tokens.

Updates

I’ve now managed to implement a SpaceEngine Format file to use as a UI layout file. In it you basically use the JSON like data format to create the individual top-level UI elements for the game.

Next is to give the UI layout files some ability to implement actions. At the moment you can design everything, but you can’t actually say what “button x” will do.

I’m thinking of separating out a “scripting” language which you can create a function in. Then in the layout file you reference that function.

This would give the UI three separate files, the Class Definitions, Layout, and Scripting. Which I think is sort of akin to CSS, HTML and Javascript.

So scripting… I’ve thought about implementing Lua or something, but like everything else, I’d rather roll my own to learn how it’s done.

My initial thoughts are to use the regular lexical parser deal as per the Dragon Book. I don’t think I need to do compiled, but just interpreted. Something like Java.

How to actually make it all function together is something of a mystery to me. I imagine the scripting processor will have set functions mapped to identifiers to be used in the script. So the script could call Game.CreatePlanet()  or something like that. The parser will see this and invoke my engines random planet spawning function or whatever.

But at what level do I abstract out these? Maybe I should abstract out the planet creation, so that the method of creation can be modded, expanded etc.?

I imagine that in future I’d want random events and AI to be scripted as well. So it seems like the lower level would allow for more flexibility.

But it also seems like it’s going to mean I need to do a lot of coding to make a full-fledged scripting language.