what to know before migrating mail from Office365 to on-premise Exchange 2019

The customer realized that due to recent events, a sudden termination of service in the MS cloud for the entire company can be expected. If only mail was spinning there, then there would be no problems: there are a lot of open-source or OS-based solutions of Russian development. But besides the mail itself, there were calendars, approvals, negotiation reservations, common boxes, complex vacation access rights and all other things. It was necessary to save not only mail, but also all the Exchange processes that the company worked with, because they were deep in its business processes.

And the customer chose Exchange 2019 in a private cloud. I know it sounds strange, but it’s a reality: there were already licenses, the stack is familiar, resources have been allocated, and the maximum that MS will do in such a situation is to deprive it of support. That the customer was not threatened, because he had already been deprived of official support. We supported it instead of MS.



These document approvals integrated with mail are one of the reasons for moving from MS to MS

In general, then it was just necessary to take it and move so that the company’s employees would not feel it. And there were several pitfalls. It started with the fact that MS, apparently, did not consider this direction of migration in principle: O365-> On-prem is simply not covered by documentation, and in the interface of the new console it is gray and inactive in the spirit of “under construction”. Well, the boxes themselves turned out to be not very small: with a normal subscription, the box receives a limit of up to 100 GB, and users confidently filled them up to about 40-50 GB each. After we migrated three boxes in a week of continuous synchronization, it became clear that something had to be done about it.


How did

We deployed four virtual machines on Windows Server 2019 Standard and installed the following Exchange 2019 roles: two pieces of Exchange CAS / Mailbox and two pieces of Exchange Edge Transport. CAS / Mailbox are, in fact, things that carry the main mail function, and EDGE are servers that receive client requests from the DMZ and transfer them to mailboxes. 2+2 is the recommended scheme and provides sufficient fault tolerance and balancing. EDGEs simply switch or transfer sessions to each other in case of problems, and databases are replicated between mailboxes, and if one goes down, you can reconnect to another instance, which one of the EDGEs will do with the user. Among other things, this means that servers can be serviced without shutting down the service.

Accordingly, a DAG cluster is configured on servers with the CAS/Mailbox role. An Exchange Hybrid configuration has been configured for mail migration and simultaneous user experience in Exchange Online and On-Premise Exchange. DAG is a balancer between mailboxes, and Hybrid is a solution so that both the cloud and the local solution can be seen in your infrastructure at the same time. This is necessary so that Active Directory does not have a difference where the objects are located. That is, during the move, user accounts were in local AD, and mailboxes – half in the cloud, half on the local server.

We had a 1 Gb / s channel to the MS cloud. But MS’s migration policy is very simple: any such actions should not slow down the system for users. That is, synchronization can only be performed at the speed that MS allows, and this is not very much. In fact, I was getting about 30-40 GB per day, which very quickly became a problem due to the size of user boxes.

It was also not possible to get user boxes with something else without special expenses: the imposed backup tools will run into approximately the same restrictions (or will interrupt the service for users), and MS makes its own snapshots in a shadow form, and there is simply no access to them from the outside.

Box size problem

When transferring a box from the cloud to local storage, roughly speaking, a copy of it is created on our installation, and then Hybrid changes addressing to local storage. The user does not notice anything, nothing needs to be reconfigured at the workplace, there is no download for support within the company.

The problem is the slow transfer. With such volumes of mail, it threatened to drag on for several months. And if a month of background synchronization still somehow looks more or less normal, then half a year is already completely overkill. It was necessary to do something with the mail, that is, to somehow clean the boxes.

Even before the migration, users unanimously said that there was not a single message in their mailboxes that could be painlessly deleted. They were cunning, of course, but it fell into the conditions of the task. Accordingly, the option “delete old mail” or “delete messages by filter” immediately disappeared.

The second proposed option was to collect local PSTs from workstations, put them in a common file dump in the form of an archive and not transfer what was in them. That is, we would get some kind of slow mail backup, but in order to get access to some old letter, we would need to connect PST to Outlook and poke around in it. The customer did not like this option either. After some thought, we came to the conclusion that it is possible to save to the archive and compress all mail, except for the last year: that is, for example, if the box lived for six years, then we put five years in the local archive, and transfer the last year live.

As a result, we decided to archive the boxes in this way, as well as cut them as much as possible in terms of technical data. If anything, MS stores a lot of deleted messages, both those that are visible to the user in the trash, and just all deleted in the entire history of the box in case the admin needs to get into the backup. You can limit the timer, for example, set storage for 14 days: after something is removed from the “swollen” box, it will be “swollen” for these same 14 days.

Actually, we disabled “SingleItemRecoveryEnabled” and set the retention time for deleted items “RetainDeletedItemsFor” to zero (as long as it is not zero, there will be a shadow copy of the box, in fact). Plus included compression. This allowed us to reduce the size of the mailboxes to migrate by a factor of ten or more.

More pitfalls

It all started simply, with a Read-Only controller on the customer’s side. For some reason, domain controllers were only in Read-Only (RODC) mode, apparently, this is how they get out of the box in some installations (although I see this for the first time). One of the domain controllers was converted to RWDC, business something.

The second feature: in the basic configuration of Exchange Hybrid, there is a problem with the main SMTP addresses of distribution groups and some users. Synchronization is done using Azure AD Connect. This synchronizes on-premises AD with Office (the cloud) so that all settings are passed both ways. One of the main parameters for correct SMTP synchronization is “proxyaddresses”. Some users and distribution groups did not have this parameter populated, and the first time they synchronized after configuring Exchange Hybrid, the primary SMTP address for these users was changed to the address of the default Azure AD organization (company.onmicrosoft.com). But it does not correspond to the real one, that is, until problem users were corrected, mail from the local infrastructure did not reach cloud users.

Now the softest part. Since MS, apparently, did not assume the reverse migration of users from Office365 to On-Premise Exchange (there is no documentation on this), the problem arose of how to link existing users, distribution groups, shared mailboxes and calendars in Exchange Online c On-Premise Exchange.

Actually, this is how this place looks in the new console:



Point gray: as if formally it is, but you can’t use it

It was done by this powershell at a time. When migrating from the new console to the classic console, there is already a migration feature.

A complete list of user mailboxes from Exchange Online has been uploaded and RemoteMailbox objects have been created on On-Premise Exchange based on them.

Some distribution groups have already been synchronized with the customer’s local AD, and objects have also been created for them on On-Premise Exchange. There were groups that only existed in the cloud. For such groups, we created similar groups in AD and rolled up settings identical to those in the cloud. After synchronization, the groups from the local AD were merged with the cloud ones. Similarly, we set up calendars so that users in the cloud see the information of users’ calendars on local Exchange and vice versa. After bringing the local Exchange in line with the customer’s Exchange Online, some of the users migrated to the local Exchange. Users in the cloud and on-premises Exchange can easily exchange mail with each other, as well as receive and send mail outside right during the process.

Total

We deployed infrastructure in a private cloud, which gives some guarantees that the service will work quietly in our difficult time. Then the migration began: outdated data was transferred to local PSTs with client-side archiving.

In total, the customer has about 1500 boxes, only about 15 TB of data after compression, archiving, deletion of the deleted ones. That is, the project is not just not small, but very hefty, but at the same time, the customer support service practically did not have to interact with users. The experience is very interesting, given the general bloodiness of the enterprise. If someone needs it, the full toolkit is already there.

Similar Posts

Leave a Reply Cancel reply