He definitely was on the more poor side. At least when Jesus was born.
When a son was born and presented to the temple, the parents would pay an offering. That was to be a one year old lamb for the burnt offering and one pigeon or turtledove for the sin offering, but there was a provision for the poor that could not afford to give a lamb to give two pigeons or two turtledoves.
Mary and Joseph gave two pigeons, which under the purification law would only be allowed if they were poor.
By the time he attended the wedding at Cana, Joseph was already dead. The last time he was mentioned was at the temple when Jesus was 12, and by the time of crucifizion Jesus entrusted Mary’s care to John. That wouldn’t have happened if Joseph were still alive.
Also, the book of John doesn’t mention any anointing oils whatsoever at the Wedding at Cana. Nothing in the passage marks it as particularly upscale.
John 2
On the third day a wedding took place at Cana in Galilee. Jesus’ mother was there, 2 and Jesus and his disciples had also been invited to the wedding. 3 When the wine was gone, Jesus’ mother said to him, “They have no more wine.”
4 “Woman, why do you involve me?” Jesus replied. “My hour has not yet come.”
5 His mother said to the servants, “Do whatever he tells you.”
6 Nearby stood six stone water jars, the kind used by the Jews for ceremonial washing, each holding from twenty to thirty gallons.
7 Jesus said to the servants, “Fill the jars with water”; so they filled them to the brim.
8 Then he told them, “Now draw some out and take it to the master of the banquet.”
They did so, 9 and the master of the banquet tasted the water that had been turned into wine.He did not realize where it had come from, though the servants who had drawn the water knew. Then he called the bridegroom aside 10 and said, “Everyone brings out the choice wine first and then the cheaper wine after the guests have had too much to drink; but you have saved the best till now.”
11 What Jesus did here in Cana of Galilee was the first of the signs through which he revealed his glory; and his disciples believed in him.
12 After this he went down to Capernaum with his mother and brothers and his disciples. There they stayed for a few days.
In fact, the master of the banquet makes mention that the normal way hosts do it is to serve what good wine they have first, then the cheapest wine once everyone is drunk, but that the host of that wedding seemingly deviated from that and served the best wine last (though they had actually run out and that was unknown to the master of the banquet). More likely is that they served all the wine they could afford already, and it wasn’t all that great of wine.
That plus the fact that Mary “was there” and instructing the servants, and then “Jesus and all of his disciples had also been invited”, it would be much more likely that it was a close family friend of Mary’s that came from more modest means.