A Non-MX mail server + Google Workspace, is this viable?

Question

I have a domain (example.com) configured for sending and receiving mail using Google Workspace. I need to launch additionally a separate (own) mail server. Switching completely to the new server is undesirable. The idea is to have some mail accounts in Google Workspace, and some on the new server.

The natural solution is probably to use subdomains (1). Nothing changes about example.com. I simply launch a mail server and link it to, say, email.example.com.

But there's another idea that struck me (2). Leave MX records pointing to Google Workspace, but change SPF to let the new server send mail using the same domain (example.com). Well, then it occurred to me that there's no way to share the DKIM's private key, is there?

If I can't configure DKIM... I don't like where it takes me. What do you think? Is it a solution worth trying, or a dead end?

What are the cons? The first one is DKIM. Also I'm not sure what happens if I send an email via the new server (using the new server's account) and the user replies. Does it go back to the new server or Google Workspace (considering that I don't want to use a subdomain)? I guess the latter. These are 2 disadvantages I can think of.

As for pros... No need to introduce a subdomain :) Okay, doesn't sound too appealing, I know. But that's all I've got.

Also I wonder if anybody tried it? I don't think I'm the first one to come up with the idea. And I'd expect people to do it in the days where spam was not widespread. (I assume there was such a time.) But these days... probably not. If they were 2 own servers (no Google Workspace), then I guess it's possible to share the DKIM's private key and then it looks like an option. Otherwise... it's probably not.

What do you think?

Answer 1

DKIM is not a problem. You don't have to share the private key, because DKIM has selectors (RFC 6376, 3.1) to support multiple email sources for the same domain. This is widely used for, e.g., sending newsletters from a different platform.

However, you would only receive mails for the domain apex with a single mail infrastructure (here the Google Workspace). If you wish to have some mailboxes elsewhere, you would have to forward mail for them. A subdomain could be used for that: the separate mail server would be receiving mails for the subdomain despite sending using the domain apex. Any replies would then travel through Google Workspace and be forwarded to the subdomain address.

Answer 2

First of all you can't create a spf record for the mailserver and then use Google workspace. You either need to include them both in the record

What you could do is create a mail relay server which forwards things to Google workspace or to your own mailserver. You can do that with postfix on linux.

I think you need to have a couple of domains because a mx record is as best practice created at the root of the domain.

Update Yes Google can forward mail to a mailbox of another user. So then GW needs a domain and your mailserver. That requires maintenance of the forwarding rules when you have a new user.

With a mail relay server, as far as I know you can forward mail for a whole domain. In my opinion better

This advice is based on 15 years of experience, i dont have a best practice document for you For best practices and pro/cons i could write a whole book about it. That involves explaining the dns best practices and mail server. You also learn that in a Linux course for example.