The short answer is no.
Recently I have had a few clients ask me this question and all are surprised to hear this answer which makes me want to shed some light on it here.
Although there is no time limit on how long you could spend outside UK while on spouse visa and it is true that your visa will not be rescinded because of your absence, there is still real adverse effect if you spend most of your time outside the UK. One of the requirements for spouse visa is to make UK your permanent home of residence. Spending more of your time outside UK makes it extremely difficult to convince the Home Office UK is your main and permanent home of residence. Of course there are exceptions to this detailed in Appendix FM.
Another adverse effect and perhaps one that most people would want to avoid is impact on application for settlement (indefinite leave to remain). According to para 245AAA.General requirements for indefinite leave to remain
The following rules apply to all requirements for indefinite leave to remain....:
(a) References to a “continuous period” “lawfully in the UK” means,.... residence in the UK for an unbroken period with valid leave, and for these purposes a period shall be considered unbroken where:
(i) the applicant has not been absent from the UK for more than 180 days during any 12 month period in the continuous period....
The impact of this is that if you spend more than 180 days in any 12 months outside the UK then is likely your application for settlement will be refused. Most people only get to know this when they are preparing to make ILR application by which point it is too late. The fact that the Home Rules on this are misleading and disjointed only make this even harder for people.
If you have any questions, get in touch with us.