Destroy one prefab clone of set of clones.

To be more specific, is there a way to destroy a certain clone from a list of clones? My problem is that I have 5 HousePrefab(Clone)s in my scene, I placed them using raycast, say I only wanted 4 and I built too many. Using this code deletes the first once built which is not what I want, I'd like to delete the one my mouse is over. if (Input.GetButtonDown ("Fire1") && collider.Raycast (ray, out hitInfo, Mathf.Infinity) && KeepingScore1.Selected == 4) { Destroy(GameObject.Find("House(Clone)")); } // KeepingScore.Selected == 4 is a reference to my 'remove' button being active. I'm a bit stuck because I'm not sure if I can use Raycast to solve my issue. Or whether I need to make an array of gameObjects or if attaching a script to the HousePrefab would do it. Any advice on the way forward would be greatly helpful.