Trouble with my group

I hostess this group on yahoo that makes ATCs. I've been doing this for a number of years, we used to have 10 or so people participating each month. At the moment I have three, including myself. I send out a package of items that fit a theme, people make ATCs with at least one of these items on their cards and send them back. The number of people has been falling a lot. Last year, we didn't swap until October. I sent everyone an ATC and asked if they would be interested in starting up again, I had 3 people say yes, and then 2, and then we got a new player in the group. Last month, I lost one of my long-time players. So, I only have 3 people playing. I sent out an email asking if people wanted to continue playing and got 2 replies, from the 2 people that are participating. I sent a reply saying I'm not doing this month because I'm not inspired. Today I got a response from one of my participants saying that when I take time off its psychologically disruptive to the players. That pissed me off. This is from one of my regular participants, who I love, and I love her work, but why can't I take time for me?
Sometimes I feel, I'm putting way more into these swaps than I'm getting back. I think up the majority of the themes (all of them right now), I'm the one putting the packets together, and sometimes I'm the one spending money on them. Lately, people have been a little short on having return postage to them, so I'm paying for that, which I can't afford to do all the time. I figure I can send them one free swap, but that's it.
She also said maybe people are dropping out because I'm taking time off. I'm not entirely sure about that. Usually when I suggest stopping for awhile, people agree. I know last year when we didn't swap, I asked that people let me know when they were ready again, but it was me who initiated the next swap and I only got a few players. It's also hard to know what people are thinking when no one replies to emails or snail mail, which leads me to believe they don't want to swap.
I was complaining to my mom a few months ago about swaps and people either not following directions, flaking out, or not getting things to me on time. She suggested I take a little time off from swaps. I really like the swaps I find online, their different and it challenges me in my art, I'm wondering if hostessing this group doesn't challenge me enough, so I find it boring.
But, then I'm told by my group that I shouldn't take time off. Art keeps me sane (that and music), so I have to do some kind of art. But, I don't just have to do ATCs, I do lots of different things, I've found hand sewing has calmed me so well lately.
I also feel I have a lot of things to deal with in life (not a ton, there are people with way more than me), but what I don't need is something else giving me problems.

I'd love some advice.

Thanks