The cat population is now larger than the human population on the island.
But Tashirojima doesnt have too many humans to begin with.
There are approximately 100 residents on Cat Island, and most of them are over 70.
One person is 37, and everybody else is over 60 years old.
The cats would go to the inns where the fishermen were staying and beg for scraps.
Sources:Wikipedia,Tofugu