Kayaking is great, but there are a few things you need to know before you head out into the water.
First things first. Know how to save yourself in case your kayak tips, and know how to use all the safety gear around. That includes the pump on your kayak in case your kayak starts to fill with water, and inflatable paddle floats, which will allow you to re-enter your kayak in the event of your capsizing out there. Make use you know all the rules of the water before heading out.
When you do head out, make sure you dress for the water temperature, not the air temperature. Quick drying clothes are best, clearly, and since there will be no shade of any variety out there, a hat and sunscreen is also a good idea. If you are not sure what to get or where to get the right stuff, Play It Again Sports is the best local water sports store around, and are a great resource for all things kayaking.
Make sure you get the right boat for you. Whether it is a ten-foot one-person kayak or a larger multi seat job (those are the ones I prefer), getting the right ride that fits both your wallet and your lifestyle will ensure the most fun you can have with your kayak.