1.) Copy the rpm to the repository directory (i.e alongside all the existing rpm's)

2.) Update the repository index:

createrepo --update  --verbose

3.) Make sure your file permissions for /repodata are ok (i.e. so that your http server can access the files):

chmod -R go+rx /repodata

(NOTE: don't forget to ensure the actual package file (the rpm) has the correct file permissions also)

4.) Clear your yum client cache:

yum clean all