Anthony - it is as easy as you said - simply change the graphics that is in the folder. re upload via ftp and it's done. Or, if you want you can use the file manager and navigate to the skin file and upload that way.
What makes it a bit more trickier would be if you wanted to change the size of the file that was managed by css - you would then need to change the css file, and if necessary, change the elements in the ascx file.
Nina