Setting up Mac OS X in VirtualBox

Mac OS X is not meant to be run inside a VM. In fact, it even violates the EULA which you have definitely read. Anyway, if you were to follow a guide for running Mac inside VirtualBox (like this) you would end up with a perfectly fine VM. However, there are some caveats. First you would notice that the resolution is stuck at 1024x768. You could fix that by changing some internal settings of VirtualBox. Next, you wouldn't be able to connect USB devices to the virtual machine because you have to install the proprietary Extension Pack first, maybe like this.

Almost there, theoretically... Since VirtualBox doesn't provide Guest Additions for Mac, you won't be able to use things like Clipboard Sync and Shared Folders. That's a huge bummer, because it's a crucial feature if you plan to use Mac and your host system side-by-side (e.g. for development). To workaround this, you can take use of Mac's built-in Screen Sharing (VNC) and File Sharing (SMB) features. But first you have to create a second network adapter in your virtual machine which acts as a host-only adapter (don't forget to create the host-only network first). This way the guest machine (Mac OS X) has access to the internet via the NAT adapter, but the host machine is still able to easily access services running on the guest via the host-only adapter. Nice!

Now on to configuring VNC and SMB. Open System Preferences in Mac OS X and go to "Sharing". Enable both "File Sharing" and "Screen Sharing". For the former you'll have to enable user authentication by clicking "Options" and check your username under "Windows File Sharing". Nothing else to do for "Screen Sharing".

You should be able to connect to the VM using your favorite VNC client now. The IP address of the Mac machine is 192.168.56.101. The same goes for SMB.

If you intend to use this setup on a daily basis, I'd recommend to start the virtual machine in headless mode, either via GUI (right-click the machine, "Start", "Headless") or via command line (VBoxManage startvm "MACHINE_NAME" --type headless). If your hardware has lots of spare resources, I'd recommend to put the VM into autostart so that you can simply connect to it via VNC whenever you need it.

Update: Well, almost. If you're using a keyboard layout other than US, you'll have lots of problems with "special characters" (-, etc). Lots of people have the same problem, but I wasn't able to fix that. Just use VMware instead.