If you have NEVER done any genealogy at all before, I suggest that you go to Cyndi's List and look for BEGINNERS. This will give you some ideas of how and where to search.
I also suggest that you try http://www.google.com
Input: Cary Genealogy
See what comes up. Contact any of these folks and see if they can connect your family lines. Can you give some dates as well as these names and where your folks were located? You will receive more hits. So many John Carys.
Try the records of the LDS church: http://www.familysearch.org
Search the Soc. Sec. Death Index for your grandfather:http://ssdi.genealogy.rootsweb.com/
There are several good Cary books that you may be able to get your local library to order for you on interlibrary loan.
1. Carey Highlights: Yesterday for Tomorrow by Virginia Miller Carey, copyright 1983. Dogwod Printing, P.O.Bo 716, Ozark, Mo 65721 contains WV/VA/KY Careys that connect to the Cary line.
2.Edward Poole of Weymouth, Mass. and his descendants by Murray Edward Poole - 1893
3. The Cary Family in America. By Henry Grosvenor Cary. Appe...
Boston, (Press of Murray and Emery Company) 1907.
Henry Grosvenor Cary, 1829-1905
Also post on the GenForum Cary message board including some places/dates.
Hope this helps a little.