disruption.ca

Reddit recently lead me to Logan Koester’s blog post about live coding with Python, and I found it fascinating. Live Coding is the practice of editing code for a running program in an environment that loads in the changes as they are made. Apparently people do this as a performance art, writing programs on the fly to play music. There is even a Wikipedia article on live coding.

I tried out the examples on Logan’s page, generated some weird drum loops, and had some fun with it all. The experience was a reminder that I want to learn how to make music. But it also opened my eyes to what seems like a new approach to programming.

After 30 seconds of looking I have failed to find a good reference on this, but I have read that a major factor in programmer productivity is the turnaround time between editing code and testing the results. If you are working on big project with a slow build process making changes takes a long time. Because changing too many things at once is a bad idea, you have to spend a lot of time waiting to test your code. If you are working on a small project in a scripting language, then it’s just save and run with little waiting to see the results of your edits.

What if we take this a step further? Are there productivity gains to be found through live coding, where you see the results of your changes when you save the code? I’ve decided it’s a neat idea, and I want to explore it. Since most of my programming focuses on game development, I tried out live coding with PyGame. What can I say? Recompiling your game’s codebase every frame strikes me as a wonderful way to trade cpu cycles for increased programmer productivity.

The first step is building the controlling script that does the live coding magic. I started with Logan’s example, but I wanted to do a couple of things differently. The first is I wanted to see a message and a backtrace for both syntax and runtime errors. The second is that I wanted the host and the client to be largely independent. Here is my controller.

controller.py:

import traceback
import StringIO

from time import sleep

done = False
imported = False
loadBroken = False
runBroken = False

def genTraceback(ErrString):
    """ Takes an error message as an arguemnt.
        Generates a traceback of the most recent
        exception, and prints it with the error
        message.  Sleeps for 2 seconds before
        returning.
    """
    fp = StringIO.StringIO()
    traceback.print_exc(file=fp)
    message = fp.getvalue()

    print ErrString,"\n", message
    print "\nFix livegame.py to continue.\n"

    sleep(2) 

# loop until we sucessfully import livegame.py
while not imported:
    try:
        import livegame
    except:
        if not loadBroken:
            genTraceback("Failed to import livegame.py.")
            # set a flag so that the traceback is only printed
            # once
            loadBroken = True
    else:
        imported = True

# loop until livegame.play() returns True
while not done:

    # attempt to reload the code
    try:
        reload (livegame)
    except:
        # if the reload fails, print the traceback once.
        if not loadBroken:
            genTraceback("livegame.py is broken.")
            loadBroken = True

    else:
        loadBroken = False

        # if the reload is succesful, try to run livegame.play()
        try:
            done = livegame.play()
        except:
            if not runBroken:
                genTraceback("play() in livegame.py is broken.")
                runBroken = True

        else:
            runBroken = False

Sorry for the \" silliness in the above code. It seems to be a wordpress bug. Anyway, what the above code does is load livegame.py and then reload it in a loop, calling livegame’s play function every time. Thanks to the traceback and StringIO modules, it outputs both syntax and runtime error tracebacks to the console seamlessly.

Here is a very basic livegame.py:

fps = 10
width = 640
height = 480

# check to see if the pygame module is in the namespace
try:
    pygame
except NameError:
    # pygame doesn't exsist, so import it.
    # this only happens once.
    import pygame
    from pygame.locals import *

    # initialize the pygame display
    pygame.init()
    pygame.display.set_mode((width,height),0,32)
    display = pygame.display.get_surface()

    # initialize the pygame clock
    clk = pygame.time.Clock()

def play():
    """ Main game logic, called once per frame."""

    # fill the display with black.  Clears the screen.
    pygame.draw.rect(display, (0,0,0,255),(0,0,width,height))

    # draw a shape.
    pygame.draw.ellipse(display, (0,40,200,255),(200,200,100,50))

    # update the display.
    pygame.display.flip()

    # check for events, and return true if we get an exit signal.
    events = pygame.event.get()
    for event in events:
        if event.type == QUIT or (event.type == KEYDOWN
                                    and event.key == K_ESCAPE):
            return True

    # try to maintain a set frame rate.
    clk.tick(fps)

    # No exit signal was recived, so return false.
    return False

The first time this code is run, it initializes PyGame and creates a window. Every time play is called, it clears the window, draws a blue ellipse, checks for events, and then sleeps long enough to try to maintain a set framerate. If you press the escape key livegame signals the controller script, and the controller exits. To see where things get interesting, change this line:

pygame.draw.ellipse(display, (0,0,255,255),(200,200,100,50))

to something like this:

pygame.draw.rect(display, (255,0,0,255),(200,200,200,100))

Save the change with the PyGame window open beside your text editor, and watch the small blue ellipse instantly turn into a larger red rectangle. A basic example, I admit. But it hints at very deep potential.

The next step is taking this live environment, and building a full game using it. I intend to give it a try, hopefully soon. It will take some cleverness to keep things both dynamic and running fast enough. I might have to set up some sort of caching system so that expensive operations, such as loading an image or level from disk, don’t happen every frame. I could perhaps bind a key so that the expensive stuff is only done on request, but I think the experiment will be most interesting if I keep that sort of thing to a minimum.

There are obvious improvements to the above code that could be made, but I wanted to keep things as simple as possible. Though I do want to add more features before I make an attempt at building something significant using the live environment. I hope to modify the controller so that it takes the name of its client as a command line argument. I would like it if the controller gave some sort of audible notification if the client code caused an exception and produced a new traceback. I will see if I can use Ipython’s handy syntax highlighting for the controller’s tracebacks. It would also be neat to modify the client code to use the python if name == '__main__' trick. Then the client could optionally run as an independent program without the controller. This would also allow changing settings such as the target frames per second based on the environment the code is running in.

If I haven’t stated it enough already this post: this approach opens up lots of interesting possibilities. The only downside is that some day I’m going to start writing flash games that run directly in the browser. When I do that, I’m really going to miss all this dynamic language magic that I’m learning to use with Python.

Edit: I’ve posted an update, Live Coding, Continued

As usual, stupid fast game development has been on my mind, so I’m going to write some of my thoughts on the topic.

Most people are quite impressed when they see what can be done in a 48 hour game programming competition. I’ve heard programmers look at competition results as say “Wow, that would take me a month.” They are correct, it would take them a month. But that isn’t because they are bad programmers, it’s because they aren’t working in the competition environment.

To better understand where I’m coming from, read this blog post about 24 hour game development sprints.

this is 100% being in the zone, where each hour spent in one of these jams is worth perhaps 10 or more hours at work in your usual environment.

I think this is very true. The right environment can give you a massive productivity boost. Here is a off-the-top-of-my-head list of reasons why the ld48 makes for such a high level of productivity:

• Super tight deadline
• Procrastination is not an option.
• You can live off of pizza, avoid chores, and sacrifice sleep because you will return to normal life after 48 hours.
• You don’t have time to second guess your decisions.
• You don’t have time for perfectionism.
• You will see results quickly.
• It’s a competition
• Competitive pressure can spur you on.
• Working along side a group dedicated to similar goals is inspiring. (Together via IRC if not shoulder to shoulder)
• It’s yours and yours alone
• You don’t have to convince anyone that it’s a good idea
• You don’t have to communicate or coordinate with anyone.
• You only have yourself to blame if it sucks.
• You’ve committed a weekend to the project, so even if it gets tough, you might as well see it through.
• No one is paying you to do it. You do it because you want to, not because you have to.

Now if only I had a way to get this level of productivity in my day to day life.

April 28th-30th was the eigth ld48 game programming competition. It’s the 4th that I have participated in. The only info online about my entry is my postmortem, which you can read if that sort of thing interests you. I’ll get around to creating a page for Bee Commander eventually, but for now I want to talk about some thoughts and observations that I have taken from this last ld48.

Following up on one of the topics from my previous post, I’ve started using iPython with very nice results. One of it’s many cool features is the ability to embed a shell into your code and launch it whenever you want. I mapped the ‘~’ key to launching a shell, and with almost no effort the development version of my game had a rough approximation of the Quake console. This goes a bit further though, because the iPython shell has full access to read and modify the entire game’s state, whereas the Quake console doesn’t know anything about the underlying engine. Also, iPython brings with it the ability to automatically launch the debugger whenever an exception occurs. Surprisingly this is less useful than I expected, as the default Python stack trace usually gives enough information to locate bugs, and the extra power of the debugger isn’t usually needed. I still haven’t really gotten into the full swing of the fabled interactive development style, where new functionality is dynamically loaded into running code and tested from the shell. But I am getting closer. I do seem to have cured myself of the need to scatter my code with diagnostic print statements every time I have a bug. iPython is a very powerful tool.

Which brings me to another topic. I initially started learning to program in Python for three reasons. It has a solid set of libraries for game development, it has solid cross platform support, and it is a high-level, powerful language. C and C++ are the standard for game programming, but I didn’t want to have to deal with the headaches of memory management plus the lack of dynamic language features. Despite more than a few frustrations with Python, I’m very happy with the choice. But I have to ask myself a question; if Python is so great for hobby game development, why are there so few good Python games? The PyGame site has lots of games on it, but very few that strike me as being really great. Every ld48 has lots of Python based entries, but they are seldom as good as the C++ games. Some neat stuff comes out of PyWeek, but I’m typically more impressed by the ld48 results.

Is it because there are better tools and libraries for C? Not from what I can tell. If there is a really useful library, there is typically a Python wrapper for it. Does Python’s comparatively slow speed hinder it? Maybe, but I don’t think so. Some of the most graphically intense ld48 games of the past might be difficult to implement in Python, but most of the best ld48 games could have easily been done in Python without having to worry about performance.

One idea that came up when Jay and I discussed this is that the talent pool for C++ game programmers is simply a lot deeper. I think this has some merit. Most professional game programmers work in C++, and when someone first approaches hobby game programming, C++ seems like the natural choice. So we have more people with more game programming experience using C++. Which would account for the best ld48 games typically being from C or C++ programmers. Adding weight to the experience matters argument is Phil. Phil is one of the more experienced Python using ld48 participants, and his mixed Python/C++ entry is the highest ranked game in the most recent ld48. As well, after working on 5 Python game projects and participating in 3 prior ld48s, I’ve gotten much better at it, and my Python game received an above average ranking. So perhaps experience is the key, and language doesn’t matter much.

If there are simply more C++ programmers with enough experience, it explains why they are typically over represented in the top of the ld48 results. I guess it makes sense, but I would have thought Python could provide more of an edge. Could Python’s edge come into play in a different manner? Ld48s often have many Python games that are ranked below average. Maybe Python makes it easier for beginners to get a game done in 48 hours, whereas you have to have a great deal of experience and skill to do the same in C++. I’ve been thinking about average rankings of C++ games vs. average rankings of Python games. It might be interesting to look at the percentage of Python users that submit a final entry, vs. the C++ users that do so.

One of the reasons that Python vs. C++ has been on my mind is because I played the ld48 #8 games so shortly after playing the PyWeek #2 games. I felt that the ld48 game were noticeably more interesting and of higher quality. Now part of this is just a numbers game, there were 66 entries in the ld48, vs. 37 in PyWeek. Plus the ld48 has the C++ programmers that I argue have so much experience, vs. Python only with PyWeek. But still, I thought that week long team efforts would produce much better results than 48 hour solo efforts could. now that I’ve thought about it, I think the extra time and the option of working in a team make it more difficult to produce something really good in a competition. It’s hard to maintain intense focus on a project for a whole week. The communication between team members creates overhead and distraction. Team entries might result in games designed by committee, which is bad when you are trying to make something great.

I read a comment in the ld48 irc channel about the distinction between good crappy graphics and crappy good graphics and I wonder if this is a factor in the ld48 producing better games than PyWeek. “I’ve got a week and a group of talented people to work with. We can create something big and complex, it will be great!” vs. “I’ve got 48 hours and I’m on my own. I’m going to make something quick and dirty and simple. It probably won’t be much, but it’s all I’ve got time for.” Perhaps it is possible to do a really good job on something limited and simple in 48 hours, but trying to make some thing big and complex is hard to do in a week. Aim as low as you can get away with and build up from there, instead of aiming at where you think you should end up.

DrPetter is a participant from the last couple of ld48s. In both competitions he made fairly simple, but very well done, platform games. The most striking thing is his art work. It looks like it was taken directly from the best Genesis or Super NES games, it’s very cool. Reading his worklog from the last ld48 I came across something I thought was interesting. He had implemented a zoom feature in his game that let him view the level from any distance, and adjust it in game. He initially created this to make debugging easier, and he was considering letting the player control the camera as part of the game play. He decided against including the feature though, because he felt that the zoom ability ruined the retro feel of the game. He has an interesting point. A Genesis wouldn’t have had the power to pull off the zoom, so you never saw that sort of thing in old games, it’s a modern feature made available be the power of 3d accelerated video cards. If he had included the feature, his game would have felt more modern, and players would probably more naturally compare it to modern games, which a 48 hour game can’t compete with. It would have crossed the good-crappy/crappy-good line. As the game stands though, it is very cool with an excellent 16-bit retro feel.

So, the moral of today’s story. I like Python for game programming, and I’m liking it more as I gain experience with it. Also, good-crappy is way better than crappy-good. If that makes any sense at all.

The typical post to this blog is either a minor update about my life or a mention of the availability of some bit of code that I’ve been working on. This post I’m going to try something different. I’m going to ramble a bit about programing and a problem that is on my mind. You have been warned.

In my experience, when writing a game I end up with a lot of constants in my code that determine how the game plays. The values for these numbers are typically not derived by any sort of logical process, instead they arise through a guess and test process that is apparently known as tuning in the industry. I think that the quality of a game’s tuning is one of the major factors in determining if it is any fun. Of course I want to make my games more fun, so I come to the question of how do I improve my tuning?

Should I be more rigorous and put more effort into the logic of the numbers that I chose? To some extent I think that I should. It would be nice if I could look at the numbers in my code and actually understand roughly what they meant in terms of units that I use in the real world. With the extremely rapid game development environment that typically drives my projects, it is far to common that the numbers are pure magic. So I should work on that. But I think there are limits to the benefits of this approach. If I take it to it’s extreme, I’m trying to build physically accurate simulations. While good physics can be a lot of fun in a game, there is a big difference between a racing game and an accurate car simulator. I’m more interested in making a fun racing game.

How then, do I make game engine tuning more effective? I’ve been asking google this question and came across one very useful tip in a Gamasutra article on physics tuning (bugmenot required). Integrate live parameter tweaking into your game, so you can try different values as you play, without having to edit the source and restart. I picture a python object that I can somehow initialize with my constants, which would then allow me to tweak values live in game, and store the results back to a config file. All with minimal pain. I discussed this with Jay a bit, and he had some interesting input. He mentioned that while working on Steamrollers he sometimes would load a level, test it, back out to the menu, edit the engine source code, and then load the level again to see the changes, without ever fully exiting the program. As well, he thought that the place to interact with constant tuning would be the terminal window, not the main game window. This is contrary to my initial thoughts, but it makes a lot of sense.

So I want to use the terminal to interact with my running program, load changed source code on the fly, and test out tweaks to constants while I play the game. I think that this sort of dynamic program interaction is starting to sound a lot like what Lisp people talk about with the Lisp shell environment. Fortunately Python has it’s own shell, and it is apparently quite powerful. Ipython sounds interesting as well. The only downside is that I really don’t know how to use the tool effectively, or if it is even possible to do what I want within the framework of PyGame. So now I’m reading a lot about the Python shell. Or at least I was until the whim to update my blog hit me.

I’m quite intrigued by this path. It should be good to learn some more powerful tools than just the text editor. I keep reading about the power of Python’s introspective abilities. I hope that learning this sort of thing will make me a better programmer. I tend to focus too much on the power and ease of use of a language, and completely ignore any tools that could save me time or solve problems for me. I think that it’s time that I stop missing out. Or at least finally break the habit of using print statements instead of the debugger. Changing habits and learning new tricks will probably be a hard slog, but I’m enthusiastic. At least for the moment.

So hopefully with some research and some practice I’ll soon be tweaking my gameplay using all sorts of insane ninja Python dynamic language-Fu. Which should help make my games more fun. I hope. But I wonder. One of the problems that I encounter is that after a while I become too close to the code and it becomes very difficult to tell if the game is enjoyable to play. Fun and playability are a hard thing to judge. In fact I think there is another whole rambling bloggy essay in that last sentence alone. I’ll leave that for another day though, and try to stick with tuning.

What if I could test and measure the results of trying different game engine tunings? When Valve developed the first Half-Life they had playtesters play their levels. Valve recorded all sorts of useful statistics, and set up rules to tweak their level designs. If the numbers showed that players were going for long stretches without firing their weapons, monsters were added to relevant parts of the levels. If the players health was always very high or very low, the level difficulty was adjusted. If a player took a very long time to get past a puzzle, a hint might be added to make the puzzle less frustrating. If you want to know more, the details come from this Gamasutra article. (again, bugmenot is your friend.)

Unfortunately I’m never going to be able to do this sort of playtesting for a 48 hour game project. But I will have to spend some time thinking about the possibilities of having friends try my games, and getting usability information that would let me tune the game appropriately. Of course there is the standard feedback. “It would be better if you could move faster.” “I can’t control it.” “Your game sucks.” This sort of thing can be valuable and helpful. But what I would really like to see would be some numbers that pinpoint the problem areas. I just don’t know if it is doable in a general, reusable sort of way, or if I would have to write measuring tools on a case by case basis.

I don’t seem to be coming up with anything sensible at this point, and I’m getting tired, so I’m going to leave this thought here. Making a game measure the amount of fun it’s player is having seems to be a non-trivial problem.

So, this was a fun exercise from my point of view. I updated my blog. I enjoyed writing it. I explored some ideas that are on my mind. Fun. I might have to try this approach again. My only question is, how does it read?

Another game programing competition has come and gone. This time it was PyWeek. Make a game in a week using python, with no limits on team size. A fun experience, but wow, doing “crunch mode” game programming for a week is not easy. It was excellent to have a wide range of people involved with the project, and it lead to things like good art, and more than one level.
If you want to give it a try, it’s a big download:

• Source: steamrollers-bugfix.zip
• Mac (Tiger Only): steamrollers-bugfix.dmg.gz
• Windows: steamrollerswin32-bugfix.zip

Also, take a look at the SnV entry page.

I’m quite happy with what we produced, but there are a bunch of quality improvements that I would like to get to now that we are outside of the one week timeframe. As always, finishing something is surprisingly hard. Especially without the pressure of a deadline.

I have a renewed interest in my last ld48 game. Hopefully I’ll find the time to put some work into it soon.

On an unrelated topic Jay and I had a discussion of the relative merits of different keyboards as weapons. He tells me that his generic plastic 105-key is the ultimate in keyboard weaponry. Silly boy. A quick blow to the head from a Model M would teach him the error of his ways.

I’ve written some basic documentation based on what I have learnt about game programming with Python. If that is your sort of thing, please check it out. Feedback is always appreciated.
Examples Of using PyGame and PyOpenGL for 2d game graphics

I really should fit the above documents into the blog page structure. Other things that are on the bloggy todo list:

• Create a page for datacam, my latest ld48 entry.
• Create a page for evilboxes. Which has a minor update: evilboxes-1.1-Win32.zip
• Replace the template. I like WordPress’s default template, but it would be nice to have something less common.
• Get a real server. There has been talk about colocating a real server for a while. I need to get off my but and make it happen.

In other news, I’m looking forward to being on vacation next week. 3.5 days in Toronto then 4.5 days in Montreal. Should be fun.

Also, I’m really looking forward to the PyWeek programming competition on the 26th. It’s about using Python to make a game in one week, with no restrictions on team size.

Last Monday Steve and I wrote a game.

Steve wanted a randomized version of the Escapa! game. So we made Evil Boxes.

evilboxesWin32.zip Windows version.
evilboxes_src.zip Source.

I have added a png2css page to the site.

Here is an example of an image rendered as css boxes.

Last weekend I participated in another ld48. I’ll set up a page for my new game at some point, but by popular demand, I’m posting a link.

It’s kind of like Katamari Damacy in 2d. Use the mouse like you are rolling the ball with your hand. Collect small stuff to get bigger before you run out of time. If you run out of time, the game exits.

Windows version:
datacamWin32.zip

Also, you can play Jay’s game online here:
http://gaiaswrath.wardtek.ca/

I’ve added Rejeweled to the games.