Mobile CRM and Epicor Web Access - Security/Best Practice

I am in the process of building the security program at my company and I can certainly empathize. I’ve also learned that security is a spectrum not a destination. Reducing blast radius, reducing surface area, and zero trust are all parts. We are choosing to adopt the NIST cyber security framework for critical infrastructure, which is something I doubt these fuel providers did (and justifying even more the entire reason for the csf for critical infrastructure).
The whole point being, it’s less about reducing windows (anyone else tired of the analogy yet) and more about the strategy around it. There are perfectly valid and secure ways to integrate over the internet. There are also ways to be breached even if you don’t have an exposed web server. With the proper strategy in place for the 5 categories of the NIST csf, one stands a very good chance of building a robust and resilient environment

5 Likes

I am in the midst of the CMMC certification process (built upon the NIST 800-171 framework that I assume you are referring to) to be compliant with DHS contractor and subcontractor requirements. We have a lot of CUI (controlled unclassified information) along with a lot of IP to protect. I realize not everyone is in this situation, but for context that is where I am coming from. Downtime for us is very expensive, as is the potential loss of IP.

Sorry for wearing out the “windows” analogy for you Aaron, but thank you for contributing to this thread.

2 Likes

It seemed like this thread was on the verge of going off the rails and becoming too heated but everyone kept civil and it ended up being a good conversation. Good job everyone :clap:

3 Likes

Intellectual discourse is critical for us all!

3 Likes

Thank you all, it has been a great discussion. @Mark_Wonsil and @MikeGross chimed in that they would use an application proxy. @aaronssh uses a very segregated approach to ensure maximum security as his costs of being down and losing proprietary info are extremely high. @josecgomez spoke about the need to utilize the internet and noted the many corporations that are using ssl and IIS to the best of their ability… it would be nice to hear more about 2nd factor authentication with Epicor and if anyone is having success using that with mobile CRM yet. He also mentioned the access scope for API.

I was really hoping that between at @aaronssh 's take on this and maybe some examples of attacks that he has, and @josecgomez ‘s approach (as well as others’ opinions who have opened up websites that integrate with their app server) I could understand “the pros and the cons of each approach and take that which you are comfortable with.”

I just do not see a very “defined” approach. What approaches are there? Is this also too broad of a question, so broad that I am going to get broader responses back? @aaronssh gave a pretty concrete example of his approach, but I haven’t seen any others chime in with how they have approached integrating with CRM or exposing their app server. Can anyone speak to that? Did you just use port 443 and an SSL cert and call it a day or what else is there? What are we fighting against?

I am leaning more towards what @josecgomez said because there are pros and cons to each decision and to speak only in absolutes about security will take some of your technological capabilities off of the table, the same capabilities that could give your company a competitive advantage (think CRM, Ecommerce, etc.), but at the same time could risk increasing the attack vectors and cost you money.

In short, it would be nice to know what the “approaches” are as @josecgomez said in his post. Are there any typical approach paths that someone can speak to or lay out? An external link to someone else’s approaches for web apps would suffice too. Really hoping for a flow chart/architecture chart to visualize these approaches, but hey I am not picky, I like to read too. It would help to understand these approaches if we can define them and then list out the pros and cons. I just don’t have the understanding of cybersecurity to answer these questions and was hoping to find someone who does. Any contacts are much appreciated.

Thank you all again.

1 Like

We use second factor via Azure AD. We authenticate Epicor with Azure Active Directory and forced second factor on there. So to login to Epicor if you are in our internal network it uses SSO, but if you are coming from anywhere external you need to authenticate against AD and provide your second factor, works great and right out of the box.

AppProxy is great approach that has already been recommended.

We wrote a portal that allows customers to access their data we did it via a Proxy on the DMZ which sits between them us and our network.
That server can only communicate with Epicor via port 443 (our Epicor server is not exposed directly) I covered this in more detail in a prior post… (let me look for it)

here you you

Which is basically similar to what AzureAppProxy does

1 Like

With Epicor SaaS, it just worked. :man_shrugging:

1 Like

@josecgomez - curious as to how did you did this? the appserver looks for AD auth… how did you split it up for internal vs external - different appservers and different config files (or URLs now)?

Nah if you run Azure / ADFS you can respond via DNS differently. External points to plublic endpoint while internal points to internal ADFS.
Think Internal Auths to 10.xx
While External Auth: azure.microsoft.com (or whatever)

If you’ve logged in to the workstation with you azure credentials (via ADFS) then it is smart enough to know you are.

That’s where this little magic happens

Click Login and it just knows who I am.

2 Likes

Thank you!

@josecgomez Thank you sir! We’ve not explored this too much since we’re not really doing it yet, but rather trying to get it all prepped for the 700/2021.x path. This will give us something to look at.

The nice thing about Azure AD Conditional Access is that when Azure AD MFA is down (which seems to happen every few months), the people in the building can still log in without MFA - if you so choose.

2 Likes

This showed up in my mailbox today:
Modernize Your Identity Management System | Microsoft Security

1 Like

I just want to add some relevant facts from a live situation that is taking place now.

On Patch Tuesday this month, Microsoft patched a bug in IIS. IIS is Microsoft’s website hosting engine. It is used by Epicor on our internal network but is also the method which the developers often ask an admin to expose for external access.

This bug is rated 9.8 severity out of 10 on the formal CVSSv3 scale (CVE-2021-31166). All an attacker has to do is send a malformed packet to the IIS server and then they can execute malicious code at the kernel level (full admin access). It its security advisory Microsoft said the bug could be used to create network worms that jump from server to server and recommend “prioritizing the patching of affected server”.

So this is exactly the situation that I was trying to describe before. If the website is hosted on IIS and our Epicor is IIS, a worm of this nature could have full admin access to the web server and then immediately jump to full admin access of the Epicor server. From there they can exfiltrate data and/or launch ransomware. It’s way too easy for an attacker to get in.

Showdan is a search engine used by hackers. Basically one can go there and say, give me a list of IP addresses for all servers using IIS, then feed that data into a ransomware module using this bug and they’re off on their way. That a server is using IIS is not something that can be hidden, it is public information. We also have to consider that China or Russia knew of this bug and were using it several months before Microsoft caught on, that is often how these things go.

This is a perfect example of why I have a hard “no” on exposing our production ERP server to outside entities. Our firewall keeps us isolated from these things and we are best protected not opening up firewall ports to our internal servers.

Thanks for the article! Do you have a link to anyone talking about this vulnerability and how they are defending against this?

Does an azure application proxy help defend against this?

What tools are out there to stop these attacks? People must be stopping them somehow, right?

as he stated it was revealed in a patch Tuesday (meaning it’s been fixed before it was out) a lot (most) of these are responsibly disclosed by researchers and fixed before they can cause damage. There are zero days out there sure but the majority of these are immediately addressed by the company.

As long as you are keeping your systems up to date you are probably okay assuming you are also applying all other best practices.

1 Like

Here is a list of vulnerabilities in popular firewalls, and some of these CVEs are 10.0 out of 10.0.

CVE security vulnerability database. Firewall Products (cvedetails.com)

This is the very point of Zero Trust and Defense in Depth is trying to make. Prevention gets one to about 90%. You still must assume breach. Like Aaron says, “It’s way too easy for an attacker to get in.”

The most dangerous attackers might be our own employees. Not malicious mind you, but they make mistakes. They open emails they shouldn’t. They get hit by drive-by malware on legitimate sites. There was a campaign of attackers mailing (via UPS) USB drives and Teddy Bears to company executives.

This is where people are using Zero Trust and Defense in Depth. Start with knowing what are the assets that you must protect. Move to identity: who are my users? Which machines are mine? Which applications do we know about? Now create a matrix around the protect-surfaces with rules about who, what, when, where, and why a resource accesses that asset. Microsegment these assets to reduce the risk of an intruder moving laterally. Finally, monitor. Install a canary that indicates an intruder. Log events (to an offline immutable destination) to analyze for suspicious behavior.

Google is your friend. Any video with John Kindervag, the Godfather of Zero Trust, is both informative and somewhat entertaining. Be wary of some of the articles that emphasize their product. Security is a journey and not a product.

2 Likes

Yes, firewalls do have vulnerabilities too. All of the risk we’re talking about is a game of statistics and probabilities.

Consider that web hosting is nearly a monoculture with IIS and Apache, making up 73% of the hosting market share. Compare to firewalls, where there are literally hundreds of vendors selling all kinds of different routers. If you are a nation-state hacking group and you are tasked with infecting a lot of systems, would you spend time and resources on hundreds of different routers, or would that limited time and resources be better spent on targeting something that almost everyone uses (Apache or IIS). That’s a really easy decision.

Am I worried about my routers? Not at all, they run FreeBSD and are not on the list of commonly attacked systems. But IIS, well that’s a whole different story. It gets attacked all the time.

You could use whatever web server you want as a reverse web proxy or web proxy, no IIS needed. Together with a robust firewall, IP whitelisting, updated patching, antivirus, least privileged (wow, that word is extremely hard to spell!) access, and every other security best practice, there’s a right way to do this. I don’t think anyone here is recommending putting your production server out on the internet.

Couldn’t agree more. Cybersecurity is a game of risk and reward, and each company needs to assess their risk tolerance and enact countermeasures according to the impact and likelihood.

2 Likes

From 2021 Web Hosting Statistics, Market Share, and Trends (websitesetup.org)

Server Software Market Share

We will start with the basic technology, the software that turns ordinary computers into, you know, servers. This means enabling other machines (clients) to access the data stored on them. Here, we have the following picture :

  • Apache: 36.1%
  • Nginx: 32.5%
  • Cloudflare Server: 16.1%
  • Microsoft-IIS: 7.7%
  • LiteSpeed: 7.3%

Together, Apache and Nginx corner almost 70% of the market. However, the share of Apache is steadily declining while new technologies like Cloudflare are on the rise.

In addition, Nginx and Cloudflare are more popular among top sites on the web.