Cats that have been dumped by previous owners are slower , at times, in adapting to a new environment. Quite understandably so... Who wants to invest emotions just to see them rewarded that way??.
It is now learning to cope with both a new owner AND a new environment. Has to get to know them first, so the first reaction is one of self defence.
If you re sitting, you re less of a " threat" than when you move about ( what is he/ she about to do now? Sthg against me??that s what goes on through a scared cat s mind ...)
I can t recommend enough using the pheromone based Feliway plug-in. ( not the spray, which is useful for instance for the carrier, when you move your cat or take herto the vet)
Pheromones help a cat feel " at home" in his environment. They are what a cat leaves when she rubs her cheeks against your legs or against furniture, for instance. Also, it s the resson why cats enjoy been scratched gently on the cheeks, it s as if you were saying " you re mine!"
Hope things will improve soon... A cat s sense of time is differnet to ours, keep that in mind and....be patient!
Good luck!
P.S. Am working at a book on cats behaviour...
I am quite sure he is scared. He will calm down, be sure he has some places to go and lurk or hide. Even a cardboard box with a door cut into it will do. Anywhere he can watch the family from and not feel exposed.
Don't worry, it will change. Even the most wild of ferals can be tamed, and he doesn't sound like he's in that mode.
Kitty has undergone a great life change; some adapt quickly, some take a little time.
Patient, persistent kindness will win him over.
The semi-famous Miss Teia Honey was like this when we first took her in at age 10 1/2. But she now enjoys la time, ear scritches, playing with dangling cords, even chases crumpled paper balls.
Do a search of this forum with the keyword Teia for more of her story.