Category Archives: Programming

Mojang’s new game Scrolls seems familiar…

I saw an article today about the announcement of the new game Scrolls by the developers of Minecraft and the idea looked really familiar.

Then I realized, its the same idea I started working on 8 years ago during my freshman year of high school.

The difference is they have the resources to finish it and the experience to do it right. Just another example of how having an idea isn’t nearly as important as actually creating something.

The game I made was called Dark Magic. It was a combination between a trading card game and a strategy game. You would create a deck of cards and then play those cards on a playfield. It was supposed to be Magic: The Gathering meets Civilization.

Each player started with a castle, and the goal was to capture the other player’s castle. You could use terrain to update your territory to produce more resources and build units to collect resources or fight (never got the fighting working properly though, probably the nested loops 10 levels deep).

I never quite completed the game so it has a lot of bugs in it, but here is a link to it anyways: http://ra726.net/DarkMagic.zip.

It is a .exe, so I doubt many people will want to run it, but I promise it is clean, no viruses or anything. Install it in a VM if you don’t trust it. Just make sure to set your screen resolution to 1024×768. Yes, I know, you shouldn’t need to set your resolution to play a game, but when I wrote it I didn’t think anyone would be running at anything else.

It is written in Visual Basic 6 and that’s not even the worst part of it. I had just started programming when I wrote this, so I made some really horrible design decisions.

I have some great stories about problems I had when writing it, but it taught me a lot that I wouldn’t have learned until much later if I hadn’t had the motivation of getting this game working.

Its amazing how looking back at something you did a long time ago can show you the lessons that you read about. Finding a project to work on, creating something instead of just coming up with ideas, the importance of shipping, don’t make assumptions about the user’s system.

I got some of them right, others wrong, but hopefully those lessons will help me with my future projects. For now, maybe I will try to install Visual Studio 6 and fix a few of the old bugs.

And if you don’t want to install it, here is some screenshots of the game. The graphics were all done by my old friend Ryan Warren. Its been a long time since I have talked to him, but you can see his blog at http://psynexus.blogspot.com/.

Let me know if you try running it, I would be interested to see what anyone thinks about it.

Automatic Twitter Posting in PHP

I got a suggestion a few months ago to make FlyingPieNotifier.com post to Twitter. It is now working and you can see it posting every day at @FlyingPieNotify

Since the rest of the site was written in PHP already, I figured I would do the Twitter posting in PHP as well. I had done it before when working on webbr so I figured it would be just as easy.

But it turns out Twitter recently deprecated Basic authentication and made OAuth mandatory. This means I can’t just send my username and password, but instead needed to register as an application and go through a handshake process before I could post.

The good news is, if you are just posting from a single account and a single script, you only need to do this once.

Here is how I was able to get my server posting a name from the database to Twitter every day. For this, I assume you have a webserver running PHP that is accessible from the general internet (not on a local network only), and a basic knowledge of PHP.

Step 1: Register as an application

To use OAuth with Twitter, they require you to register as an application. This makes sense if you are writing an application to deal with multiple users, but even for 1 you have to do it.

If you go to http://twitter.com/apps you can register a new application. Most of the information here doesn’t matter for our purposes. The only thing you need to pay attention to specifically is the callback URL. For this, put a URL to a page that you can create on your server, such as http://example.com/twitter_callback.php. We will create this site soon.

Once your application is created, you will be given a consumer secret and a consumer key. We will need these in the next step.

Step 2: OAuth authentication

Now we need to go through the OAuth process to give our code access to the account. Since we only want to post for this one account, we don’t need to make this code very robust.

I used the Twitter-async library from https://github.com/jmathai/twitter-async. Grab the EpiCurl.php, EpiOAuth.php, and EpiTwitter.php files and put them on your server somewhere you can access them easily, preferably in your include_path.

Now create a file on your server that you can access. In this file, include the files we just created, and then add this code:


$twitterObj = new EpiTwitter($consumer_key, $consumer_secret);
echo 'Authorize with Twitter';

Make sure you replace the consumer key and secret with the ones you got when you registered your application.

Now create a second file with the name and URL that you put as the callback URL when you registered your application with Twitter. In this file, put:


$twitterObj = new EpiTwitter($consumer_key, $consumer_secret);

$twitterObj->setToken($_GET['oauth_token']);
$token = $twitterObj->getAccessToken();

echo 'oauth_token: '.$token->oauth_token.'
';
echo 'oauth_token_secret: '.$token->oauth_token_secret;

Again, make sure to include the Epi files and to put in your consumer key and secret.

Now load your first file in your browser, click the link, authorize your application with Twitter, and copy down the token and token secret that are shown in your browser.

Now we have authenticated, so we don’t need to do that again! You can just delete the 2 files you created, but leave the libraries there, we will use them again.

Step 3: Posting to Twitter

Now that we have an authentication token for our account and our application, we are ready to post.

To post, all we need to do is:


$twitterObj = new EpiTwitter($consumer_key, $consumer_secret, $oauth_token, $oauth_token_secret);

$result = $twitterObj->post_statusesUpdate(array('status' => $post));
$result = $result->responseText;

You need to make sure to have the keys from your application and the secret that you got when you authenticated here or it will not work.

Now if you look at this, you may be wondering about the last line. It looks like it is unnecessary, but because of the library we are using, we actually need it. This is because it is an asynchronous library, so our call to post does not mean that the post has been made. Referencing the response of our call forces our code to block until the call has completed.

Now you just need to write some code to generate your message and run when needed. For Flying Pie Notifier, I just used a cron job that runs once a day and pulls the name from the database for that day.

After this, there is a lot more you can do with these PHP scripts. The documentation for the Twitter-async library shows how to use the rest of the Twitter API in addition to posting.

Sending Email To Gmail From PHP Without Being Marked As Spam

While working on Flying Pie Notifier, I ran into a problem with Gmail marking the notification emails as spam. I was able to find the Gmail Bulk Sender Guidelines and make some changes. I think I got Gmail to finally recognize that the notifications are not spam.

My first step was to make sure that SPF was set up correctly and to check the reverse DNS PTR record for the IP address I was sending from. Both of these are simple to set up with just DNS records.

Unfortunately, those changes didn’t seems to fix the problem completely, so I had to move on to the next step, DKIM (DomainKeys Identified Mail). DKIM is a way of cryptographically signing emails that are sent to verify that they are sent from the domain they claim they were sent by.

I found a PHP library called PHP-DKIM that would perform the signing, but it was incompatible with the PEAR Mail_Mime module that I was using to send HTML email. I had to make some modifications to the code to get them to work together.

Here are the steps I used to set up DKIM signing with Pear::Mail_Mime and PHP-DKIM:

  1. Generate a public and private key pair.
    First I needed to generate a public and private key pair to use for signing messages and for other people to verify the signature. These commands generate 2 files, key.priv and key.pub, containing my new private and public keys:

    openssl genrsa -out key.priv 2048
    openssl rsa -in key.priv -out key.pub -pubout -outform PEM
  2. Publish the public key in a DNS record.
    DKIM public keys are published in a special DNS record so the recipient can look up the key and use it to verify the message signature. I followed the PHP-DKIM README and put my keys in a dkim-cfg.php file along with my domain name and a selector name and ran dkim-test.php.

    Loading dkim-test.php will tell you if you have any problems with your configuration so far, and if not it gives you a DNS record. For me, it generated this record which I added to my DNS:

    p._domainkey.flyingpienotifier.com. TXT "v=DKIM1; k=rsa; g=*; s=email; h=sha1; t=y; p=MHwwDQYJKoZIhvcNAQEBBQADawAwaAJhAODvTnvu4+OO+47m/3iRImsshlhQ+u30 0cHSdCJPndsD2meo63lNl8ELktIZKUEEEGFYNbwuZHWISCmuFHEiQNIHeag+awen 75oPBZqd1ABVdjyTwZa1matMzQhg5rSGpQIDAQAB ;"

    I then used http://www.sendmail.org/dkim/checker and http://domainkeys.sourceforge.net/selectorcheck.html to check my record and make sure it was set up correctly.

  3. Signing Code
    I had downloaded the PHP-DKIM library from http://php-dkim.sourceforge.net/, but found that the way it treated headers was not compatible with the PEAR Mail_Mime library that I was already using. PHP-DKIM expects a single header string formatted the way headers are transmitted between servers when sending email. Mail_Mime wants an associative array of headers that it then uses to generate the header string when it sends the final message.

    I decided to modify PHP-DKIM to get it to work with the associative array approach, since I thought that was the better of the solutions.

    You can find my modified version of the library at http://www.ra726.net/php-dkim.zip. The only file I modified is dkim.php. I changed to use an array for headers and cleaned up the code a bit to make it easier (in my opinion) to read.

    To make the library work with Mail_Mime header arrays, I added code to the AddDKIM method that concatenates the headers together. This is simple in PHP, and just requires:

    $headers_line = '';
    foreach ($headers as $key => $value)
    $headers_line .= $key.': '.$value."rn";

    After that, I just had to change what the function returned. Instead of returning a new header string, I just return the DKIM signature string. This can then be added to the headers using:

    $dkim = AddDKIM($headers, $subject, $message);
    $headers['DKIM-Signature'] = $dkim;

    This code could be cleaned up to automatically add the header, but it works until I take the time to rewrite the library and make it object oriented.

  4. Testing
    When I first tried to use my new code to send emails to my own Gmail account, I found that I had some errors in my implementation. In Gmail, you can see the full headers and the results of DKIM and SPF checks by clicking the arrow next to Reply and selecting Show Original. This will show you the full headers of the message, including the verifications added by Gmail.

    When I did this on my message, I got an error telling me the body hash was wrong. The body hash is used to verify that the email message was not tampered with during delivery. Since I had sent the message myself, I knew it had not been modified. Therefore, Gmail was calculating the hash based on a different version of the message than my current code.

    I was able to find a service that could help me track down the problem. At http://www.port25.com/domainkeys/ you can find information about an automated responder that will let you know the results of the DKIM verification.

    By sending my message to the Port25 reflector test address, I got a report letting me know all the details of the DKIM verification process. One of the pieces of information in this email was the canonicalized body of the email. Canonicalization is the process of standardizing the message to specific requirements to make sure both sides are doing the hash calculation on the same data. I could then use the NiceDump function in PHP-DKIM to print out the canonicalized message that I was calculating the hash on. This let me see where I had caused a problem when modifying the library.

    Using the same testing address, I was able to find a few other problems with my modifications to the library and get it working. Once I had the bugs worked out, the reflector address reported the DKIM verification as passing. It worked!

There is no definite solution for not having your emails marked as spam. All you can do is try to follow all the guidelines that Gmail and other providers specify. It seems to me that Flying Pie Notifier is no longer being marked as spam, so I think I should be good for now. DKIM is a part of that, but I also had to make sure my mail server and DNS records were all set up correctly.

If you want a really detailed technical explanation of DKIM, you can find the specifications in the RFC at http://www.dkim.org/specs/rfc4871-dkimbase.html

If you have any issues with this code or are trying to implement it yourself, leave a comment and I can try to help you out.

– UPDATE –
Originally this post used a 384 bit private key. This key size is trivially broken now, so I have updated it to specify a 2048 bit key.

Introducing FlyingPieNotifier.com

Introducing my latest website project: Flying Pie Notifier.

If you aren’t from Boise, Flying Pie is an amazing pizza place and has been my favorite for years. In addition to having amazing pizzas, they have something called It’s Your Day. Each day they have a different person’s name. If it is your name, you can come in and make your own pizza.

I did It’s Your Day a while ago and had a lot of fun. But I always forgot to look online and see when my name or my friend’s names were coming up. So I decided to make a website to remind myself.

Right now, you can go to FlyingPieNotifier.com and sign up. Just enter your email address and the name you want to be notified about and you will get a notification sent to you when your name is coming up. You can also sign up to get a weekly reminder of the It’s Your Wednesday group (and maybe some other things soon).

I am working on adding in some extra features, so let me know if there is anything you want. I would love to hear any comments or suggestions.

Arudino and 1-wire

For my birthday this year, my wife got me the Arduino Start Kit from Adafruit Industries.

If you aren’t familiar with Arduino, it is a great open source platform for doing small electronics projects. The start kit I got has a Arudino Duemilanove, which is the latest basic version of the platform. The Duemilanove has a small processor on it that can be easily programmed over USB, and a lot of input and output pins to play with.

After getting the kit set up and my prototyping board soldered together, I decided that my first project would get seeing if I could get some old 1-wire temperature sensors working. A few years ago, I got a sample of 5 of these sensors but was never able to get them working with my homemade serial adapters. The sensors I got are the DS1820 from Dallas Semiconductor. These are old and have now been replaced by the DS18S20, but they work the same.

Turns out that the Arduino community had already done all the hard work at getting these working, so it only took me a few minutes to get it set up.

Hardware: Each of these sensors has 3 legs: power, ground, and data. If you are only using a few sensors that are close to the power supply, you can instead use parasitic power as I did. This reduces the number of wires to the sensor needed to 2, data and ground. To do this, I shorted the power and ground pins and wired them to ground. I then attached the center data pin to one of the digital IO pins of the Arduino.


Here you can see the Arduino with the sensor connected on the prototype shield from the starter kit.


This is a close up of the connections. The brown wire connects the power and ground pins. The black wire connects these to ground. The yellow wire is connected to digital I/O pin 2 on the Arduino.

Software: The Dallas Temperature Control Library is a library that allows simple control of the 1-wire temperature sensors. All you have to do is download it, put it in your Arduino libraries folder, and open the Simple example.

The example sets up the library and reads the current temperature from the first sensor, printing it out to the serial monitor.

After years of having these sensors sitting around in a drawer, it only took me a few minutes to get them working. Now I just need a larger project to use these in.

XNA Index Buffers on Laptops

I have been working through The Hazy Mind XNA tutorials at http://www.thehazymind.com/XNAEngine.htm.

The computer I have been using for development is my old laptop that has a Intel 915 GM. This card has integrated pixel shading, but relies on the CPU for vertex shading. It also has some other issues, but I won’t go into those now.

As I went through the tutorials, I was having issues rendering the textured quads that are the first object made and rendered. These quads are also used to create the skybox, so getting them working was required to continue developing through these tutorials.

These two pictures show the issues I was having. The first picture shows a single triangle not rendering correctly, and the second shows two triangles not rendering.

Int Render Error 1

Int Render Error 2

After quite a few tries, I finally discovered what the problem was. In the tutorial code, the Index Buffer is created using:


int[] inds = { 0, 1, 2, 1, 3, 2 };

myIndices = new IndexBuffer(myDevice, typeof(int), 6, ResourceUsage.WriteOnly, ResourceManagementMode.Automatic);

myIndices.SetData(inds, 0, 6, SetDataOptions.None);

In this, each of the indicies is represented by an integer. For some reason, using ints causes the rendering errors. By looking at other tutorials, I found that many of them used shorts instead (A short is just an integer that uses less memory and stores a smaller range of numbers).

To fix the errors, all I had to do was change the code to:


short[] inds = { 0, 1, 2, 1, 3, 2 };

myIndices = new IndexBuffer(myDevice, typeof(short), 6, ResourceUsage.WriteOnly, ResourceManagementMode.Automatic);

myIndices.SetData(inds, 0, 6, SetDataOptions.None);

Here is the final result. You can see that the rendering problems are gone.

Short Render

Moving From DirectX9 to XNA

I’ve been working the last week on getting started with XNA and how to port AquariusSim over to it. Learning XNA has been easier than I expected it to be since I had already written a lot of code in DirectX9.

Microsoft did a good job with most of XNA, but there are a few things that I think will really frustrate new developers. Getting started has been improved a lot. It now only takes a few lines to set up a device and render a blank screen. That was one of the things that took me the longest when I was starting to learn DirectX. Most of the complexity of setting up the device has been hidden, which makes it a lot simpler. It might end up making some advanced features harder to implement, but I am not sure because I never used any of them.

Switching to fullscreen is also a lot easier. When I was working on the first version of AquariusSim I remember spending days working on getting that to work, but its now just a few lines of code.

The one thing I like the least about XNA is the removal of the Fixed Pipeline. The Fixed Pipeline in DirectX allowed you to render objects using simple Lighting Objects and not have to worry about learning another programming language to implement shaders. XNA has removed this functionality and requires shaders to be written for everything. This means that the programmer has to either learn a completely new langauge, or they have to just use other people’s shaders without understanding how to change them. This could end up being a serious block to new game developers.

Overall, I think Microsoft made some really good decisions for XNA and I think that for someone experienced with 3D graphics it will be an easy switch. But as before, there are some serious roadblocks that could make aspiring game developers give up before they get very far into it.

Now, back to working on this port. Hopefully I should have some screenshots up soon and maybe even some code examples of what I am doing.

Aquarius Simulator XNA

If you talked to me a few months ago, you will know that I was working on a game using C# and DirectX. The last few months have been pretty busy so I haven’t worked on it in a while, but I am going to start trying to convert it over to XNA.

XNA is the new version of managed DirectX and will allow me to (once I get one) port the game over to the XBox360 pretty easily. Since Microsoft is moving all of their development over to XNA I figure it will allow me to avoid work trying to move over to it later.

I am going to be learning XNA through some tutorials by the same person who I got a lot of the code for the first version from. You can see the tutorial on his site at http://www.thehazymind.com/XNAEngine.htm. As I go through those, I hope to also start my own series of tutorials or at least walk-throughs of how my development is going.

For those of you who don’t know, the game is going to be a simulator similar to the SimCity line of games. The difference is that I am basing it on the idea of Aquarius from the book The Millennial Project. This means that the game will consist of managing a floating island city.

I will be posting more about this as I continue working on it, so look for more soon.