Instances of OpenLayers.Marker are a combination of a OpenLayers.LonLat and an OpenLayers.Icon.
Markers are generally added to a special layer called OpenLayers.Layer.Markers.
Example
var markers = new OpenLayers.Layer.Markers( "Markers" );
map.addLayer(markers);
var size = new OpenLayers.Size(10,17);
var offset = new OpenLayers.Pixel(-(size.w/2), -size.h);
var icon = new OpenLayers.Icon('http://boston.openguides.org/markers/AQUA.png',size,offset);
markers.addMarker(new OpenLayers.Marker(new OpenLayers.LonLat(0,0),icon));
markers.addMarker(new OpenLayers.Marker(new OpenLayers.LonLat(0,0),icon.clone()));
Note that if you pass an icon into the Marker constructor, it will take that icon and use it. This means that you should not share icons between markers -- you use them once, but you should clone() for any additional markers using that same icon.