Yes it's entirely possible you have a vacuum leak inside the brake booster. It has a rubber diaphragm inside and can dry rot and crack over time. You can disconnect the brake booster vacuum hose and plug the hose with something that cannot get sucked in, like a bolt or rubber cap that fits snug over the hose. Then start the engine and see if it stays running at idle. That will isolate the booster at least. Do not try to drive without the booster, you will have very little brakes. Often times the failure is just the hose fitting grommet though.
You mentioned the brake are spongy, that can be from multiple issues including bad brake fluid, it should be almost perfectly clear. When it gets dark it's a sign it has absorbed moisture. Moisture compresses, brake fluid does not. You may also have air in the brake lines and calipers. Flushing the brake fluid is low cost and pretty easy to do with 2 people and will eliminate air and bad brake fluid. It will also give you an opportunity to inspect the brake lines and hoses for any leaks.
BTW, the engine is the vacuum source, the brake booster is vacuum operated, but has no ability to produce vacuum.