In the WTF folder, the Config.wtf, look for SET realmname "server name", replace which ever server is in parenthesis with the server of your choice and save the file.
Make a back up copy of the Config.wtf before you mess with it and don't have the game running while you change it or it won't work.