What's the Difference Between .bashrc, .profile, .bash_profile, …?

Basically, ~/.bash_profile is loaded when you login. It is read only once. ~/.bashrc is loaded everytime you start a shell (⁖ starting a terminal), but is not loaded when you login the first time.

Here's a excerpt from man bash:

/etc/profile
     The systemwide initialization file, executed for login
     shells
~/.bash_profile
     The  personal initialization file, executed for login
     shells
~/.bashrc
     The individual per-interactive-shell startup file
~/.bash_logout
     The individual login shell cleanup file, executed when
     a login shell exits
~/.inputrc
     Individual readline initialization file

There's also {.login, .profile, …}. These are basically legacy that bash may also read.

• The .login is legacy login init file for the original shell, the Bourne shell “sh”. BASH (Bourne-Again SHell) is a mostly compatible extended version. On many unix systems, /bin/sh is hardlinked to /bin/bash. In Ubuntu Linux, 2006, /bin/sh is a sym link to /bin/dash. dash is new. “dash” is smaller and faster than bash, for running system startup scripts.
• The .profile is legacy shell init file for Bourne shell “sh”.
• The .csh is init file for “csh” C shell. (basically, shell with C syntax.)
• The .tcshrc is init file for “tcsh” tcsh. (basically, improved csh)

Note: on Mac OS X, the Terminal app starts as a login shell, so it runs ~/.bash_profile. Different unixes have different setup. You should not worry about “correct” init file too much; just try and test on each system.