Automating Phishing Simulations for New Employees in M365 — Sort Of

0 Comments 10 min read

A few weeks ago we tried to improve our phishing simulation game for new employees in Microsoft 365. The goal was simple: new hires should automatically receive a phishing simulation email after their first one to two months on the job — just to make sure they know that phishing exists and that the IT security team might occasionally test them. Easy, right? Spoiler: not quite.

Automating phishing simulations for new employees in Microsoft 365 using Attack Simulation Training and Entra ID dynamic groups

Key Takeaways

  • Microsoft's Attack Simulation Training can target new hires automatically, but only by way of a dynamic group — there is no native "new employee" trigger.
  • Driving the dynamic group off employeeHireDate is more accurate than using createdDateTime or whenCreated, which often predate the actual start date.
  • If you sync from on-premises Active Directory, employeeHireDate has to be fed from a custom AD attribute such as msDS-cloudExtensionAttribute1 via Entra Connect.
  • Simulation automations cannot be scheduled in advance, so the workflow is still partly manual — typically one prep session per month.
  • Attack Simulation Training requires Defender for Office 365 Plan 2, which is included in Microsoft 365 E5.

Environment

Before we dive in, here is what we are working with:

  • Microsoft 365 tenant with Defender for Office 365 Plan 2 (included in M365 E5).
  • All users have an M365 E5 license.
  • On-premises Active Directory synced to Entra ID via Entra Connect.

The Problem

Microsoft's Attack Simulation Training in the Security portal does offer an automation feature. The idea is that you define a condition, and when a user matches it, they automatically get added to a simulation. You would think "new user joins tenant" is one of those conditions. It is — but only if you define a dynamic group and point the automation at it. Fine, we can do that.

Here is where it gets interesting. You need to figure out which attribute to base the dynamic group query on. The obvious candidate is createdDateTime (or whenCreated in on-premises AD), which reflects when the user object was created. But that can be misleading — user objects are sometimes created weeks or even months before the employee actually starts. Not ideal for targeting someone fresh on their first day.

Fortunately, Entra ID has an attribute for exactly this: employeeHireDate. By default it is not populated, but we can work with that.

The Solution

Step 1 — Populate employeeHireDate via Entra Connect

Since we are running an on-premises Active Directory synced with Entra Connect, we can map a custom AD attribute to employeeHireDate in Entra ID. We used msDS-cloudExtensionAttribute1 for this. It is one of several cloud extension attributes available in AD and, unlike most standard attributes, it is free to repurpose.

For existing users, we wrote a short PowerShell script to set the attribute. Since we do not always have the exact hiring date on hand, we used whenCreated plus one month as a reasonable approximation: 

Import-Module ActiveDirectory

Get-ADUser -Filter * -Properties whenCreated, msDS-cloudExtensionAttribute1 | ForEach-Object {
    $hiringDate = $_.whenCreated.AddMonths(1).ToString("yyyy-MM-ddTHH:mm:ss.0000000Z")
    Set-ADUser -Identity $_.SamAccountName -Replace @{
        'msDS-cloudExtensionAttribute1' = $hiringDate
    }
}

For new users, the same logic was added to the existing interactive PowerShell user creation script. Ideally, someone manually fills in the real hire date — but let's be honest, that does not always happen. The approximation is good enough for most cases, and is better than nothing.

If you want to know how to map msDS-cloudExtensionAttribute1 to employeeHireDate in Entra Connect, check out this post: Configure EmployeeHireDate and EmployeeLeaveDateTime in Active Directory to be used with Microsoft Entra ID Governance. Microsoft's own reference for the attribute lives in the Microsoft Graph user resource documentation. After a successful sync, you should be able to see employeeHireDate populated on the user objects in Entra ID.

Step 2 — Create a Dynamic M365 Group

With employeeHireDate now populated, we can create a dynamic membership group in Entra ID. One important note: it must be an M365 group or a mail-enabled security group. Regular security groups do not work with Attack Simulation Training. Yes, that is just how it is.

The dynamic membership query filters users whose hire date falls within a specific window. Rather than hardcoding dates that need monthly updates, we let Entra do the math using system.now with ISO 8601 duration offsets: 

(user.employeeHireDate -le system.now -plus -p30d) -and (user.employeeHireDate -ge system.now -plus -p60d)

Breaking that down:

  • -le system.now -plus -p30d — the hire date is at least 30 days in the past, meaning the employee has been around long enough to receive a simulation.
  • -ge system.now -plus -p60d — the hire date is no older than 60 days, so we are not sending a "welcome" phishing mail to someone who joined two years ago.

The result is a window that moves forward automatically every day — no manual date changes needed. It is worth noting that employeeHireDate is still in preview as of writing, so treat it accordingly. Once the rule is in place, use the Validate Rules feature in the Entra portal to test against a few users with known hire dates before going live. It takes maybe two minutes and saves you from a potentially awkward conversation later.

Step 3 — Configure Attack Simulation Training

Now we can set up the simulation in the Microsoft Security portal under Attack Simulation Training → Simulations. Use your newly created dynamic group as the target audience.

Here is the catch we ran into: simulation automations cannot be scheduled. Microsoft's automation feature in this context does not let you set a future start date. This means we cannot set up a fully automated "fire and forget" workflow for new hires. Instead, we prepare one simulation per month in a single session and let them run on schedule. It is a bit more manual than we hoped, but it is manageable.

A few things to keep in mind when setting up each monthly simulation:

  • Only use payloads in your company's primary language, unless you intentionally want to test multilingual awareness.
  • Set start and end dates to align cleanly with the calendar month so the reporting is easier to read.
  • Choose payloads that are realistic for your industry — a fake invoice or HR notification works better than something obviously suspicious.
  • Review the results from previous months before selecting new payloads — some patterns get stale and click rates drop for the wrong reasons.
  • Make sure the landing page and training assignment match the payload theme so the learning moment lands.

Frequently Asked Questions

Do I need Microsoft 365 E5 to run Attack Simulation Training?

Attack Simulation Training is part of Defender for Office 365 Plan 2, which ships with E5 and is also available as an add-on for lower SKUs. You do not strictly need every user on E5, but the users you want to enroll do need the Defender for Office 365 Plan 2 entitlement.

Can I use a regular security group instead of an M365 group?

No. Attack Simulation Training only accepts M365 groups and mail-enabled security groups as targets. Plain security groups will not appear in the picker. This is the single most common reason a dynamic group built for this purpose fails to work as expected.

Why use employeeHireDate instead of whenCreated or createdDateTime?

Object creation timestamps reflect when IT created the account, not when the employee actually started. In environments where accounts are provisioned weeks or months ahead of the start date, whenCreated targets the wrong window. employeeHireDate is the attribute Microsoft specifically intends for this purpose.

Is employeeHireDate generally available yet?

At the time of writing, employeeHireDate in Entra ID is in preview. It works reliably in our environment, but its behavior, supported scenarios, and Graph API surface can still change. Validate the dynamic rule against known test users before scaling up.

Can I fully automate the monthly simulation rollout?

Not today. The simulation creation step itself does not support scheduling a future start date through the automation feature. The dynamic group keeps itself current, but creating and launching the next simulation is still a manual click-through. A monthly batch session is the practical workaround.

Conclusion

Is this the cleanest solution? No. Should a platform like Microsoft 365, with its automation features and extensive user attribute support, handle this natively without requiring a chain of attribute mappings, dynamic groups, and monthly manual setup? Probably. But here we are.

The good news is that it works. New employees get a phishing simulation shortly after joining, employeeHireDate gives us a more meaningful trigger than raw object creation time, and the monthly prep sessions take maybe thirty minutes once you have a template to work from. Not elegant, but effective — which, in IT, is often the best you can hope for.

Related Posts

0 comments:

Post a Comment