I recently acquired the following info on the poor house as my GGG Grandfather died there in 1884.
hope this helps:
Information about the Poor Farm is sketchy at best. It was something not written about that much.
The census have a separate listing and from what I can tell they were listed by the time that they were institutionalized. The Poor Farm is in the Town of Orangeville.
The Poor Farm changed over the years. Sometimes people came and went, sometimes they stayed until they died. I even found one person who seemed to wander from Poor Farm to Poor Farm.
The earliest Poor Farm was where the indigent could be taken care of. Towns were assessed money for the care of every inmate from their Town. (A Town in NY is a subdivision of a county; a village is where the grocery store and post office are located.) A section of it was the mental asylum.
Later on the stigma of the Poor Farm lessened and you read in the paper how someone had sold their house and were going to live at the Poor Farm as a paying customer..... basically assisted living. In the later years it was basically a care facility. In the papers you read how some church group entertained at the Poor Farm. I think it closed in the 1950s or so. It is now a huge dairy farm.