Simulate Attacks in Attack Simulator Requires MFA But I Already Have It?

“You must enable multi-factor authentication (MFA) to schedule or terminate attacks. Learn more about enabling MFA.

I wanted to play with the Attack Simulator in the Office 365 Security & Compliance Admin Portal – but with the enabling MFA warning, none of the ‘Launch Attack’ buttons were available to use. I’ve already set up MFA via Conditional Access though, so why am I seeing this?

At a guess, I wondered if it was actually detecting if MFA was used to log in. It wasn’t because the request was coming from a trusted IP address, which I’d configured in my test tenant to make it a bit less painful.

My hunch was right, I signed in elsewhere, went through MFA and look, the buttons now work:

Bit of a misleading warning – your MFA rules might be completely fine, so try signing in with MFA first before going to the Simulate Attacks page.

User Can’t Receive MFA Requests for Azure AD / Microsoft 365

Was stumpted on this one and had to get advice from Microsoft Support.

A single user couldn’t log in via Multi-Factor Authentication. SMS code would say it was sent, wouldn’t come through. Phone call also wouldn’t come through. Trying to set up another MFA method would receive one of these errors:

You are blocked from performing this operation. Please contact your administrator for help.

We’re sorry, we ran into a problem. Please select “Next to try again.

There were zero search results for that first error word for word, which is never a good sign.

There’s several areas you can check for blocked users such as:

But I couldn’t find the user listed in any of those.

After logging a case, Microsoft Support advised to check here:

And of course, that’s where the user was listed. They’d had some suspicious activity (a MFA phone call they didn’t initiate) so chose the option to block future sign in attempts, as you’d hope. This also triggered an email alert to admins, and that link is where the user’s block is listed until released.

Small Business Data Breaches in Australia and My Experience with One

What happens when a company you deal with in Australia has a data breach, and their annual turnover is less than $3 million? I thought I’d find out, after this happed to me. Here’s the events in chronological order with some information censored:

The Dropbox Email

I receive an email from my Strata Management company in October 2019 whom I’d already regarded quite low in their digital actions – emailing without unsubscribe options, using email addresses given to them purely for Strata related comms for commercial purposes – but this was more concerning again:

To me this was immediately dodgy and cried out of an account being compromised. The file is still there right now, ~ 6 months later. It’s a standard jump file – redirecting you off to phish your creds.

The credential stealing page you get to from the ‘Access Document’ link above is down – the entire domain doesn’t respond, so at least nobody will get caught by this link.

I email back with what I thought was correct information from a quick Google on it, but the details are their problem to work out and send an email to the address listed on the website, rather than the compromised account:

From: Adam Fowler
To: admin@company

Hi,It appears XXX account has been compromised by a third party, which includes my personal contact details on it. You’ll need to comply with the government’s Data breach standards:

Under law I believe you have 30 days to disclose this breach:

Adam Fowler

Two weeks pass… nothing. I follow up:

From: Adam Fowler
To: admin@company, person@company


Any chance of getting a response on this?

Adam Fowler

I get an out of office from the person, but it doesn’t take long for their manager to respond:

From: Manager@company
To: Adam Fowler

Hi Adam,

Thank you for your email and I apologise for the delayed response – the front office thought it may have been another scam email due to the multiple links and opted to delete and ignore it.

As you are aware, we do have 30 days to respond to this with the breach happening 15 days ago we still have time on our side. In saying that, we actually have our IT guys coming in today again to assist me with the lodgement and I will be finalising it either tonight or over the weekend.

I can confirm we acted promptly on the issue and our IT guys responded extremely fast as well.

Thank you for your concern and notification, I will confirm with you once this has been lodged.

Have a great weekend.

This sounded sort of promising – beyond the weird conclusion my email was another scam, they seemed to be treating this seriously and properly. I was content with this and waited for the confirmation that was promised.

That confirmation didn’t come, so 1 month later I followed it up. This is where it went downhill:

From: Manager@company
To: Adam Fowler

Hi Adam,

I did lodge this and I spoke with the Office of the Australian Information Commissioner.

As far as I am aware from them there was no further action required from us on their end.

Kind regards,

OK… that’s great that they’ve met legal requirements, but that’s not really what I cared about:

From: Adam Fowler
To: manager@company

Hi Manager,

I’m more concerned if any of my personal data was compromised after your investigation rather than what data breach notification steps you’ve taken with the government?

Adam Fowler

Another concering response:

From: Manager@company
To: Adam Fowler

Hi Adam,

No personal details have been compromised from this. They did not have access to our server.

Kind regards,

This takes me to a conclusion pretty quickly that they really have no idea what they’re talking about, or just trying to get rid of me because I’m a hassle. I call them on it:

From: Adam Fowler
To: manager@company

Hi Manager,

That’s obviously incorrect, my email address is personal information, and XXX’s mailbox may have contained other personal information that I’ve emailed them, such as the address of my unit.

Apparently what I’ve asked for hasn’t processed and they’ve given up:

From: Manager@company
To: Adam Fowler

What would you like for me to do Adam, I’m not sure on what steps you are asking me to take? 

This got me annoyed. I have no idea what data they have on me and what could have potentially been accessed, so I did a bit of research and shot off what I wanted, outlining why I was concerned:

From: Adam Fowler
To: manager@company

Advise on what data of mine was actually accessed. “None” isn’t true or I wouldn’t have received a phishing email. The responses you’re giving don’t give me any confidence that you’ve actually had this investigated, or have any reasonable understanding of the statements you’re making. My next step is to lodge a complaint with the OAIC, which I’d rather not bother to do.

You hold money that is partly mine, my personal details and I’m not sure what else.

Separately, I’ll actually request you provide a copy of all personal information you hold on me, as per

Access your personal information — OAIC
Australian privacy law gives you a general right to access your personal information.This includes your health information.. An organisation or agency must give you access to your personal information when you request it, except where the law allows them to refuse your

Please let me know what other details you need from me for this request.

Three days later I get this answer:

From: Manager@company
To: Adam Fowler

Hi Adam,

Below is all of the information we have for you.

Salutation: Mr Fowler

Mr Adam Fowler

*My home address*

*my mobile*

*email address different to the one they’ve sent this email to*

We don’t have your bank details and as I mentioned, they did not have access our server so they would not have received the above information.

I’ve searched XXXs emails over the past two days and you do reference your unit, but never your home address.

It was obvious they weren’t doing this properly. They didn’t list the address of the ACTUAL PROPERTY they managed for me, nor the email address they’d just emailed me on. I decided to just stop responding and lodge a complaint with OAIC; I didn’t really have anything to lose by doing so. Lodging a complaint was pretty easy, there wasn’t too much info I had to provide and I included the email thread above.

The next day after filling in the form, I received a fairly generic email which contained the case number I’d been given:

From: OAIC
To: Adam Fowler

Dear Adam Fowler

Thank you for your correspondence received on 2 December 2019. The Office of the Australian Information Commissioner (OAIC) has registered this matter as a privacy complaint by you about STRATA MANGEMENT COMPANY

We aim to contact you further about your complaint as soon as we are able to. Information about what happens to your privacy complaint is available on our website,

Actions you can take now

·      Generally for us to consider your complaint you first need to have complained to the respondent. While waiting to hear from us, we recommend that you continue to pursue resolution of your complaint with the respondent organisation.

·      You may also be able to lodge your complaint with a recognised External Dispute Resolution (EDR) Scheme. A list of recognised EDR schemes is available on the OAIC’s website. These EDR schemes cover financial services (including credit reports), telecommunications, and energy and water providers. If the OAIC considers your complaint would be more effectively or appropriately dealt with by a recognised EDR scheme, we may decline to investigate the matter.

·      If your matter relates to consumer credit, please forward a copy of your credit file to this office, as well as copies of any correspondence you have received from the credit provider, credit reporting bodies and any dispute resolution body you have complained to about this matter. You should also include the relevant password if the copy of your credit file is password protected.

Next steps

Unfortunately we are not able to allocate all complaints to a case officer as soon as they are received. At present there are delays on some matters being allocated because we have had an increase in the number of complaints we have received.

At this time, it may be several months before an officer contacts you about your matter. We will contact you earlier if we are able to.

Once your complaint is allocated a staff member will contact you to discuss the next steps in our complaints handling process. The OAIC aims to resolve privacy complaints by conciliation, whereby the parties resolve the matter through discussion and negotiation. Unless we consider it inappropriate to do so, your complaint will likely be referred to the respondent for it to contact you directly to try and resolve the matter.

Please let us know if your contact details change, if the matter has been resolved directly with the respondent or if other circumstances change.

You can write to us or call on our Enquiries Line on 1300 363 992 (local call cost, but calls from mobile and pay phones may incur higher charges). If you do contact us it will help us if you quote your complaint reference number which is found at the top left hand side of this correspondence.

We will arrange for letters and telephone calls to be translated if you would like to communicate with us in a language other than English. You can also let us know if you need other assistance, including documents in other formats or larger fonts.

Yours sincerely

Enquiries Team

Office of the Australian Information Commissioner

That didn’t give me much hope, so I left it at that and moved on.

2 months later, I received a call on my mobile. It was from the OAIC who had started to review my case. We had a chat, she understood the situation, completely agreed they hadn’t appeared to have done their due diligence in the data breach or provide me with my personal data as requested.

It sounded promising and I was a bit nervous. Their standard approach was to talk to the company and somehow come to an early resolution. She emailed me what was discussed too:

From: OAIC
To: Adam Fowler

Dear Mr Fowler

I refer to your privacy complaint about STRATA MANAGEMENT COMPANY , made under s 36 of the Privacy Act 1988 (Cth).

I am conducting preliminary inquiries under s 42 of the Privacy Act. The purpose of the inquiries is to establish whether this matter can be resolved quickly by the Early Resolution Team.

The Early Resolution team aims to resolve matters within 4 weeks. If the complaint cannot be resolved by 28 February 2020 and the OAIC determines further review or investigation is required then the matter will be referred to an investigations officer in another team.

If the matter is referred to another team, it can take several months to be allocated to a case officer. We therefore encourage both parties to try and resolve the matter through this early resolution process.

Next steps

We have provided a copy of your complaint to STRATA MANAGEMENT COMPANY and requested it provide the OAIC with a response to your allegations and to your proposed resolution.

We have also invited to contact you directly to try and resolve this matter. In our experience, direct contact between the parties leads to a higher chance of resolution.

We have requested STRATA MANAGEMENT COMPANY provide an update in a week’s time.

I am happy to discuss this matter and to clarify any questions you may have about our Early Resolution process. If you have any questions, please feel free to contact me directly on XXX or email to

Yours sincerely 

Investigations Officer
Dispute Resolution Branch

The same day though, my hopes of anything were completely shot down:

From: OAIC
To: Adam Fowler

Dear Mr Fowler

In my conversations with  STRATA MANAGEMENT COMPANY it appears it is a small business operator and may therefore not have any obligations under the Privacy Act 1988 (the Privacy Act).

The APPs apply to businesses and not-for-profit organisations with an annual turnover of more than $3 million and to all private health service providers irrespective of turnover.

I have asked  STRATA MANAGEMENT COMPANY to respond to questions to confirm it is a small business operator and to provide evidence of their turnover or a statutory declaration.

If  STRATA MANAGEMENT COMPANY is a small business operator we will be unable to take any further action in the matter. I will write to you to let you know if this is the case along with our intention to decline to investigate the matter.

I was rather confident this company didn’t turn over $3 million a year. However, the manager did still call me and after advising he didn’t have to respond legally. I didn’t really say much since I had no legal standing now and in the laws eyes, they were in the right. They attempted to reset the password so I could access my own data from their systems – he couldn’t get that working so I did a password reset myself. Their password reset process actually sent me an email that contained my old password in plain text – ‘dontsendthisout’ – which I’d set a few years ago after they’d sent me my password in plain text via snail mail, along with the username and login URL. As I said at the start, I didn’t expect much from this company.

The data they had on me they said, would all be in this app. Again this of course isn’t true because of the data in their emails, but I felt defeated and didn’t press on this.

It was of course confirmed that they didn’t turn over $3 million a year:

Dear Mr Fowler

I refer to your privacy complaint about STRATA MANAGEMENT COMPANY, made under s 36 of the Privacy Act 1988 (Cth) (the Privacy Act).

The Office of the Australian Information Commissioner (OAIC) conducted preliminary inquiries into your complaint under section 42 of the Privacy Act.

I have reviewed your complaint and I do not consider there has been an interference with your privacy on the basis that  STRATA MANAGEMENT COMPANY appears to be a small business operator. The reasons for this view are explained below. You now have an opportunity to comment before I make a final decision.

Small business operator exemption

The Australian Privacy Principles (APPs) in the Privacy Act cover many private sector businesses in Australia, but there are exceptions. In particular, many small businesses are exempt from the obligations outlined in the APPs in the Privacy Act. Under the Privacy Act, a small business operator is a business with an annual turnover of $3 million or less that:

·      is not a health service provider

·      does not trade in personal information

·      is not a contracted service provider for a Commonwealth contract

·      is not a credit reporting body

·      is not related to a body corporate that carries on a business that is not a small business

·      does not operate a residential tenancies database.

In response to our inquiries,  STRATA MANAGEMENT COMPANY provided information, including its Business activity statements (BAS) to establish that its annual turnover and activities are such that it meets the Privacy Act’s definition of a small business operator.

This means that  STRATA MANAGEMENT COMPANY is not covered by the APPs in the Privacy Act and therefore there can be no interference with your privacy under the Privacy Act through STRATA MANAGEMENT COMPANY’s actions in this instance.

Next steps

Section 41(1)(a) of the Privacy Act gives the Commissioner the discretion not to investigate a complaint if she is satisfied that the act or practice complained about is not an interference with privacy, as defined in the Privacy Act.

As STRATA MANAGEMENT COMPANY appears to meet the Privacy Act’s definition of a small business operator, I intend to decline to investigate your complaint under section 41(1)(a) of the Privacy Act.

However, before I make a final decision I invite you, should you wish to do so, to provide a written response to this email. I would appreciate receiving any response by 11 March 2020. If I do not hear from you by this date, the OAIC will make a decision based on the available information and close your complaint.

If you would like to discuss your complaint, I may be reached XXX during business hours, or email

Yours sincerely

Investigations Officer
Dispute Resolution Branch

I briefly responded saying I couldn’t dispute their annual turnover, and the act is the act.

The final emails redeemed themselves a bit, when the CEO emailed me without further prompt:

Dear Adam,

I understand you have made a complaint in relation to the dropbox email that was sent out when XXX’s email was hacked. As you were not satisfied with our responses, I have contacted my IT team and asked them to email me an explanation of what happened and what would have been hacked.   Please find below an email from our IT Company.   YYY is happy for you to contact him directly if you need, but I would ask that you cc me in on any email.  I have not copied him into this email to protect your email address.

I have inserted his email below.

Hi Adam,

It is our understanding that the breach was caused by XXX clicking through a link in a scam email and it tricked her into putting in her email password. As such that gave the hackers access to her Office 365 based email account. Once noticed, that day, we changed her password and confirmed they didn’t have access anymore.

It did not give them access to any other email accounts, though we changed all passwords to be sure anyway.

And it did not in any way give them access to the server where STRATA MANAGEMENT COMPANY store files and run their management databases. The server is not linked to Office 365 at all, and even if she used the same password for 365 as her PC/server then it wouldn’t matter as she didn’t have remote access allowed on her account, and our remote access also requires a certificate that the hackers didn’t have access to. So I am certain they never had access to the server.

Subsequently there has also been no signs of any breach of the server or anything further on her email account.

So in short you can be assured that only her email was breached.

As for what they did access or download from her email I cannot say, we can’t tell that from the logging available in 365. It seems unlikely to me they did download information. The usual thing with these hacks is they use the compromised account to perpetrate another scam to force a bank transfer. I’d say that they worked out she wasn’t responsible for bank transfers and so instead used her account to try to hack more email accounts.

So the only data that they could have about you is anything you emailed to XXX, with the exception of anything she deleted after you sent it and before they hacked in.

Let me know if you want any more information.

I felt that at least they’d now had a better understanding as to what happened, and MAYBE cared a bit more about the impact of it.

From: Adam Fowler
To: CEO@comany


Thanks for the additional details and the explanation makes sense. I’d also expect they’d do basic searches for things in an account like credit card information and bank details which is why I was asking what XXX may have had in her inbox in relation to me.

The other question is why you didn’t have MFA in place on your Office 365 accounts – easy to do and protects the data that I send your company from these threats. I hope you’ve implemented it since, as it’s a relatively easy setting to turn on.

Adam Fowler

The CEO thanked me for this email and said they’d pass it on to their IT department. I hope they’ve actually implemented MFA now as it seems their external IT support is reasonable, and I wouldn’t expect a smaller company to have advanced Office 365 logging features available in an E5 plan to see what was accessed exactly. They’re still the company that holds the money for the Strata pool of funds, so I care that our money isn’t stolen.

Finally, the OAIC closed the case:

Hi Adam

Thank you for your emails and feedback on the Office of the Australian Information Commissioner’s (OAIC) 26 February 2020 view that there had not been an interference with your privacy on the basis that STRATA COMPANY, is a small business operator.

As a small business operator, STRATA COMPANY does not have to follow the Australian Privacy Principles (APPs), so it does not have to provide you with a copy of your personal information, or follow any of the other APPs in relation to security, use or disclosure of your personal information in the Privacy Act. It may have other legal obligations in relation to how it handles personal information.

I acknowledge your concerns and view that regardless of the technicalities of the Privacy Act 1988 (Cth) (the Privacy Act), your privacy has been breached.

However, as defined in the Privacy Act , an interference with privacy can only occur when an APP entity breaches an APP in relation to personal information about the individual (section 13). As STRATA COMPANY meets the definition of a small business operator in the Privacy Act, it is not an APP entity which is subject to the provisions of the APPs in the Privacy Act, and it cannot interfere or breach your privacy as specified in the Privacy Act.


Section 41(1)(a) of the Privacy Act gives the Commissioner the discretion not to investigate a complaint if she is satisfied that the act or practice complained about is not an interference with privacy, as defined in the Privacy Act.

As STRATA COMPANY is exempt from the provisions of the APPs in the Privacy Act, I have decided under s 41(1)(a) of the Privacy Act not to investigate the complaint on the grounds that there is no interference with your privacy as defined in the Privacy Act.

The file is now closed.

Thank you for bringing this matter to the attention of the Commissioner. I am sorry we are unable to assist you.

Yours sincerely

Investigations Officer
Dispute Resolution Branch

Although I could say that nothing happened out of this 5 month experience, I hope it was a valuable lesson for the staff there – and the CEO knows a bit more about it.

Blocking ActiveSync with Conditional Access

Microsoft has announced that they’re continuing the path away from Legacy Authentication, with the decommission of legacy auth to EWS on Exchange Online on October 13th 2020. Instead of waiting for that looming date, there’s a bunch of security reasons to only have Modern Authentication for Microsoft 365.

I’ve already written up on Protect Your Office 365 Accounts By Disabling Basic Authentication and Blocking Legacy Authentication – Conditional Access vs Authentication Policies – but when I migrated from Authentication Policies to Conditional Access, I didn’t realise ActiveSync wasn’t included as part of blocking Legacy Authentication, even though it connects without MFA.

The guide from Microsoft on how to block Legacy Authentication doesn’t actually mention ActiveSync, so it’s easy to miss like I initially did! You’ll need to block ActiveSync altogether as far as I know, as it doesn’t support MFA.

Although I still think Conditional Access is easier to manage than Authentication Policies, there is one caveat; even with an ActiveSync block in place via Conditional Access, too many attempts by a user will lock their account briefly. This might cause problems or require work to get those users to clean up whatever device is trying to log in. With an Authentication Policy I don’t believe this happens because it’s blocked earlier in the sign-in process – you won’t see logs, and the account can’t get locked.

There is of course, a checkbox around ActiveSync, and a way to block it using Conditional Access, but I had mixed results in blocking it successfully until I did it exactly this way:

Create a new Conditional Access Policy and set these options:

Users and groups > All Users
Cloud apps or actions > Select Apps > Office 365 Exchange Online
Conditions > Client apps > Tick both ‘Mobile apps and desktop clients’ + ‘Exchange ActiveSync Clients’
Grant > Block Access

In the Users and Groups section, you can narrow this down from ‘All Users’ for testing or for a gradual rollout.

The user experience is interesting on this one – they can still sort of authenticate, but instead of getting their emails, they will see a single email advising that their access has been blocked:

On top of this, you can use Azure AD to audit who might be using ActiveSync before you put any sort of block in place. As per usual, there’s a good Microsoft article on Discovering and blocking legacy authentication which can walk you through this, but in short:

Via the Azure Portal, go to Azure Active Directory > Users. Under Activity, go to Sign-ins. Click Add filters, and choose Client App > Tick the three ‘Exchange ActiveSync’ options and press ‘Apply’. You’ll see the last 7 days of sign in attempts using ActiveSync, which should give you an idea of how many users are using it, and who.

Blocking Legacy Authentication, plus blocking ActiveSync will give you a much more secure environment, protecting from account attacks.

Blocking Legacy Authentication – Conditional Access vs Authentication Policies

I’ve already written a post on why Legacy Authentication (Basic) is bad, and Modern Authentication is good. At the time of writing, Authentication Policies were the way to go to block Legacy Authentication methods. Of course, things change and there’s now a better* option to look at – Conditional Access.

I’ve also covered Conditional Access before, and it’s really hard to fault the solution. There are now Baseline policies deployed by default (still in preview though) to Azure AD tenants with recommended best practices:

Conditional Access Baseline Policies

One of these is for blocking legacy authentication – but I’m not going to recommend you turn this on (at least for starters, it’s good at the end when you know you have full modern authentication support), as it’s a tenant wide setting that has no exceptions if you need to allow legacy authentication for an account (unlike Require MFA for admins, which does allow exceptions).

Instead, you can create your own policy that does the same. This means you can gradually roll it out, and put exceptions in place until you either work around them, or live with them. If you have a requirement for an account that requires legacy auth, then you need to consider how else you’ll protect that account – can you use other Conditional Access policies to restrict it to a certain region/locations, certain apps, platforms etc – lock it down as much as you can, and make sure the account has a long unique password.

The single important setting to block legacy auth via a Conditional Access Policy is blocking access to ‘Other clients’ via Client apps:

Microsoft have a full guide on how to set this up on

So, why is this better than using Authentication Policies? Two main reasons:

If an account has their access or signin blocked due to an Authentication Policy, it’s not logged. You can look at the user in Azure AD and check the sign-ins, but you won’t see anything. However, if it blocked via Conditional Access, you’ll have a nice log entry showing you it was blocked:

Side note: Although in this example I was logging in from Australia, I was trying to connect to Exchange Online via PowerShell. That seems to often be detected as being in the US, so be careful with region blocking.

The other reason is that Authentication Policies can take up to 4 (!) hours to apply, although it’s often more like an hour. That is a long time to wait, and you just have to keep waiting and trying until it works – except if you did it wrong, you won’t know and you’ll keep waiting. Or, if you need to unblock access while rolling out, it’s a long time to roll back.

Authentication Policies do have their place though, they give more granular control over what you want to block or not – say you know you want to block POP3 access company wide, but not IMAP – that’s possible in there, but not via Conditional Access.

Unless you have a good reason to use Authentication Policies, just use Conditional Access (and assuming you have Azure AD Premium P1 or P2 licensing to actually let you use Conditional Access, and if you are using Azure AD you should be on that licensing anyway). It’ll make your life easier!