State of LLMs

Survey conducted by CentML and DevAI

Introduction:

This survey aims to capture the evolving landscape of Large Language Models (LLMs) within various companies, examining developments in tools, applications, and challenges in deployment and operationalization.

Your insights will contribute to a comprehensive report on the current state and future directions of LLMs and its adoption. Afterward, if you wish to leave your contact information, we will share the results and findings of this survey with all participants.

Instructions:
Please answer the following questions based on your organization's experiences and strategies. Estimated completion time is 7 - 9 minutes.

Question Title

1. Which industry does your organization operate in?

Technology

Finance

Healthcare

Manufacturing

Retail

Other (please specify)

Question Title

2. Please describe your organization's size.

Fewer than 10 employees

10 - 50 employees

50 - 100 employees

100 - 250 employees

250 - 1,000 employees

1,000 - 10,000 employees

more than 10,000 employees

Question Title

3. What is your organization’s level of familiarity with Large Language Models (LLMs)?

We are just exploring LLMs.

We have experimented with LLMs.

We use it in everyday business processes

We have fully integrated LLMs in production for internal workflows and applications

We have fully integrated LLMs in production for customer-facing applications

Question Title

4. If you are considering implementing LLMs, which operational improvements are you targeting? (choose all that apply)

Automating repetitive tasks and processes

Generating insights from data to improve decision making

Streamlining customer interactions

Expanding content generation capabilities

Improving employee productivity with co-pilots

Employee re-skilling or learning and development

Revenue growth

Cost reduction

Other (please specify)

Question Title

5. How critical is LLM adoption in your organization's strategic planning?

1 (Not critical)

6 (Extremely critical)

Question Title

6. How do you plan to use LLMs? (Choose all that apply)

Self-hosting LLMs in private data centers

Self-hosting LLMs in a cloud vendor - - AWS, GCP, MS Azure, OCI

Use hosted LLMs from a cloud vendor - AWS, GCP, MS Azure, OCI

Use hosted LLMs from a model provider - OpenAI, Anthropic, Cohere, others

Question Title

7. What integration challenges do you foresee in incorporating LLMs with your existing data systems?

Compatibility issues

Data privacy concerns

Integration complexity

No major challenges

Question Title

8. Do you plan to use any of the following techniques on your proprietary data?

Fine-tuning

RAG

None of the above

Unsure

Question Title

9. What factors are most important to your organization when selecting an LLM solution provider?

Performance and reliability

Cost and pricing flexibility

Compliance and security features

Support and partnership capabilities

Ease of deployment

Other (please specify)

Question Title

10. Does your organization have sensitive data that cannot be sent to cloud-based LLMs?

Yes

Not applicable

Question Title

11. Does your organization have the necessary infrastructure to support high-performance computing for LLMs?

Yes, we have sufficient accelerator-powered infrastructure.

Yes, but we are upgrading it.

No, we need to upgrade our infrastructure.

Unsure

Question Title

12. What is your current level of investment in LLM adoption(monthly amounts)?

Exploratory budgets (<$10k)

Moderate investment ($10k-$50k)

Significant investment (>$50k)

Large Scale investment(>250K)

Question Title

13. Do you have in-house expertise for managing LLMs, or do you rely on external services?

In-house expertise

External consultants

Managed services

None

Question Title

14. Have you encountered challenges due to hardware limitations in deploying LLMs?

Yes

Unsure

Question Title

15. Do you have specific concerns regarding the cost of deploying and maintaining LLMs?

Yes, costs are a major concern.

No, costs are manageable.

Unsure

Question Title

16. How critical is response/query processing latency in your LLM-driven applications?

Critical

Moderately important

Not a priority

Question Title

17. What timeline do you anticipate for your organization to deploy large-scale LLM use-cases?

Already deployed

Within 6 months to 1 year

Within 1 year to 3 years

3 years or more

N/A not deploying

Question Title

18. Are there particular LLM models you are considering for your use cases? (Select all that apply)

Llama

Mistral/Mixtral

Anthropic

OpenAI GPT-4

Google’s Gemini

Other (please specify)

Question Title

19. How do you plan to manage the lifecycle of AI models within your organization?

In-house development

Using third-party tools

Not planned

State of LLMs

Survey conducted by CentML and DevAI

Question Title

1. Which industry does your organization operate in?

Question Title

2. Please describe your organization's size.

Question Title

3. What is your organization’s level of familiarity with Large Language Models (LLMs)?

Question Title

4. If you are considering implementing LLMs, which operational improvements are you targeting? (choose all that apply)

Question Title

5. How critical is LLM adoption in your organization's strategic planning?

Question Title

6. How do you plan to use LLMs? (Choose all that apply)

Question Title

7. What integration challenges do you foresee in incorporating LLMs with your existing data systems?

Question Title

8. Do you plan to use any of the following techniques on your proprietary data?

Question Title

9. What factors are most important to your organization when selecting an LLM solution provider?

Question Title

10. Does your organization have sensitive data that cannot be sent to cloud-based LLMs?

Question Title

11. Does your organization have the necessary infrastructure to support high-performance computing for LLMs?

Question Title

12. What is your current level of investment in LLM adoption(monthly amounts)?

Question Title

13. Do you have in-house expertise for managing LLMs, or do you rely on external services?

Question Title

14. Have you encountered challenges due to hardware limitations in deploying LLMs?

Question Title

15. Do you have specific concerns regarding the cost of deploying and maintaining LLMs?

Question Title

16. How critical is response/query processing latency in your LLM-driven applications?

Question Title

17. What timeline do you anticipate for your organization to deploy large-scale LLM use-cases?

Question Title

18. Are there particular LLM models you are considering for your use cases? (Select all that apply)

Question Title

19. How do you plan to manage the lifecycle of AI models within your organization?

Question Title

20. Looking ahead, what advancements in LLM technology are you most hopeful for to enhance your organizational capabilities?