Norman Marks on Governance, Risk Management, and Internal Audit

I find it frustrating to see so many so-called experts calling for internal auditors to grab the latest technology, the latest being generative artificial intelligence (AI), and find a way to use it.

Let me tell you this: artificial intelligence is not always that intelligent!

Let me also tell you that you should only use technology when you know how it will be used and that there will be a positive Return on your Investment (ROI).

Taking that last point first, several years ago I was invited to meet the leaders of a large internal audit department in Detroit. (I will let you guess who.) They introduced the two leaders of their new Data Analytics team, who explained that they had purchased some great tools, hired several experts in their use, and were now starting to build analytics routines. They started with Accounts Payable and then the General Ledger. But they had no idea how the rest of IA would use their reports. It was a matter of buying a great hammer and then looking for nails without thinking about what needed to be built.

Now for the less-than-intelligent generative AI.

When ChatGPT first came out, like many others I decided to download and test it. One of the first things I did was to ask it who I was. It returned a decent description of my professional history, but nothing on my personal life (and my mother[1], brother[2], and son[3] are all prestigious individuals). OK. So I tried the same question again to see if the results would be different. They were. It created a fictional version of me.

Recently, Microsoft came out with their AI, CoPilot. I tried the same test to see what I would tell me about myself. This is what it said:

Norman David Marks is a retired executive with expertise in corporate governance, risk management, internal audit, enterprise performance, and the value of information. He has also served as a coach and mentor to individuals and organizations worldwide¹. His contributions span various domains, including:

1. Auditing: Norman authored the book “Auditing that Matters: Case Studies,” which provides internal audit practitioners and students with real-life case studies for discussion and debate. These cases help upgrade understanding of world-class internal auditing practices¹².
2. Chief Audit Executive (CAE): With experience overseeing companies with up to $28 billionin annual revenue, Norman has implemented risk management, ethics programs, and disclosure processes³.
3. Businessman: There is also an English management consultant and businessman named David Mark Norman, who hails from a prominent City dynasty and is now retired from executive positions⁴.

In summary, Norman David Marks has made significant contributions to the fields of auditing, risk management, and corporate governance, leaving a lasting impact on the industry.

While it only mentions what I would call one of my lesser books, it’s not too bad.  It misses a lot of information about me that it should have been able to find easily, but at least it is me. One thing I like is that it provided references for the information it provided.

Would I recommend the tool to anybody looking for information on a potential new hire? No. It is incomplete and what is missing is, in my case, material to the decision.

I decided to test CoPilot a second time. This time, instead of asking the same question, I asked about my wife, Diana Bee Har Cheah (using her maiden name).

This is what it had to say about her:

Diana Bee Har Cheah is a distinguished Professor Dr. specializing in Breast Surgery at the University of Malaya in Kuala Lumpur, Malaysia. Her expertise lies in breast cancer management, research, and clinical outcomes. Let’s explore some of her notable contributions:

1. Association Between Cardiometabolic Comorbidities and Breast Cancer Profiles:
   • Diana Bee Har Cheah collaborated on a cross-sectional study that investigated the relationship between pre-existing cardiometabolic conditions and the pathological characteristics of breast cancer at initial diagnosis¹.
   • Contributors to this study included other researchers such as Shridevi Subramaniam, Yek-Ching Kong, Cheng-Har Yip, and Nur Aishah Taib.
2. Clinical Outcomes in Early Breast Cancer Patients Treated with Intraoperative Radiotherapy:
   • She participated in a multi-center prospective study comparing outcomes between low- and high-risk groups of early breast cancer patients who received intraoperative radiotherapy in addition to external beam radiation¹.
   • Collaborators included J. A. Mosiun, Li-Ying Teoh, Mahmoud Danaee, Lee-Lee Lai, Char-Hong Ng, and others.
3. Impact of COVID-19 on Breast Cancer Management:
   • Diana Bee Har Cheah contributed to a study examining how the COVID-19 pandemic affected breast cancer management in a multiethnic middle-income Asian country¹.
   • The research involved other experts such as Jing Hui Ng, Tneoh Jia Min, Mei Sze Teh, and Mahmoud Danaee.
4. Metastatic Malignant Thymoma Presenting as a Breast Lump:
   • She co-authored a case report and literature review on an unusual pattern of metastasis involving malignant thymoma presenting as a breast lump¹.
   • Collaborators included Kah-Seng Khoo, Kavinya Diana T. Nadesalingam, Li-Ying Teoh, Mei-Sze Teh, and Suniza Jamaris.
5. Exploring Health Information-Seeking Practices of Breast Cancer Patients:
   • Diana Bee Har Cheah was part of a cross-sectional investigation that explored the health information-seeking practices of breast cancer patients in a middle-income country with a diverse ethnic population¹.
   • The study involved researchers such as Kah-Seng Khoo, Kavinya Diana T. Nadesalingam, Li-Ying Teoh, Mei-Sze Teh, Cheng-Har Yip, and others.

Her work contributes significantly to breast cancer research and patient care, making her a respected figure in the field of breast surgery and oncology.

This is not my wife!

I showed it to her, and she was very surprised that there was someone with the same name as her.

But there isn’t!

The AI found a university in Malaysia where some on the faculty have part of her name, and others have the rest.

It generated a profile by combining the resumes of different people for a fictional professor with my wife’s name.

Now, in some situations her name is shown in a different order, with Diana as a middle rather than a first name: Bee Har Diana Cheah instead of Diana Bee Har Cheah.

CoPilot gave us this:

Bee Har Diana Cheah, also known as Madam Cheah Bee Chew, is the remarkable founder of Bee Choo Origin. Established in 2000 in Singapore, Bee Choo Origin is renowned for its herbal treatment for hair. Let’s delve into her inspiring journey:

• Mission and Vision:
  • Madam Cheah was determined to find an affordable 100% natural remedyfor common hair issues such as grey hair, hair loss, and scalp-related problems.
  • Guided by Traditional Chinese Medicine (TCM) physicians, she embarked on a journey of learning, experimentation, and concoction.
• Signature Herbal Cream:
  • After extensive research, Madam Cheah developed what is now known as the “Bee Choo Origin Signature Herbal Cream.”
  • This herbal treatment, made from premium quality herbs, has become the cornerstone of Bee Choo Origin’s success.
• Growth and Transformation:
  • From humble beginnings as a single woman working from home with a stand-alone steamer, Bee Choo Origin has expanded globally.
  • With over 170 storesworldwide, the brand continues to focus on continuous improvement, blending modern technologies with two decades of expertise.
• Core Values and Beliefs:
  • The brand’s logo reflects its core values:
    • Genuineness: Authenticity builds trust with customers and partners.
    • Excellence: Dedication to delivering the highest quality hair care solutions.
    • Care: Customers are at the heart of everything Bee Choo Origin does.
  • The flower bud in the logo symbolizes new beginnings, growth, and the power of nature to renew and restore.

Madam Cheah Bee Chew’s commitment to transformative hair care has made Bee Choo Origin a global brand, providing natural solutions for confident, healthy hair since 2000¹².

Again, MS CoPilot made a serious mistake. It found someone with a similar name and assumed (and demonstrated what ‘assume’ means) it was the same person.

So, don’t feel bad if you haven’t found a use for AI in your practice.

If you have:

1. Make sure you can trust it.
2. Make sure it will return a positive ROI.

I welcome your thoughts.

POSTSCRIPT:

Alexander Ruehle challenged me, as he should, saying that auditors should rush to AI and learn how to use it. With that in mind, I searched for pieces on the form of AI called LLM, large language models, and found this: https://deepchecks.com/top-5-risks-of-large-language-models/

This is how it starts:

While they offer great promise in terms of providing instant and accessible information, there are inherent risks associated with their potential to disseminate incorrect or false information.

One of the primary concerns surrounding LLMs is their susceptibility to generating incorrect information. These models generate responses based on patterns and associations learned from training data but lack true comprehension and fact-checking capabilities. Consequently, they may produce inaccurate or misleading information, especially when confronted with ambiguous or contextually nuanced queries. Users who rely solely on these responses without independent verification are at risk of being misled and propagating misinformation.

Furthermore, LLM and ChatGPT are only as reliable as the data they are trained on. If the training data contains incorrect or false information, the models may inadvertently perpetuate those inaccuracies. This highlights the need for rigorous data selection and preprocessing to mitigate the risks associated with training language models.

I encourage you to read and consider the rest of the article. If you want, use your favlorite AI tool to seek information on AI-related risks.

====================================================================

[1] She has an MBE for her professional and charitable works.

[2] He is a recently retired professor emeritus at Northwestern University with multiple awards and honors.

[3] He is a partner with a major law firm.