LLM and Prompt Engineering Survey Question Title * 1. What is your current occupation? Student Software Developer Data Scientist Product Manager UI/UX Designer Other (please specify) Question Title * 2. How many years of computer programming experience do you have? No Experience Less than 1 Year Experience 1-3 Years Experience 3-10 Years Experience More than 10 Years Experience Question Title * 3. Which programming languages are you proficient in? (Select all that apply) Python JavaScript/TypeScript Java C/C++ C# Go Rust Other (please specify) Question Title * 4. Which Integrated Development Environments (IDEs) do you use most? (Select all that apply) Visual Studio Code Visual Studio IntelliJ IDEA PyCharm Eclipse NetBeans Other (please specify) Question Title * 5. Are you aware that you can develop AI applications based on LLMs such as ChatGPT? Yes, I've heard of that. Yes, I have experience with that. No, I know nothing about that. Question Title * 6. What is your development experience with LLM-based applications? No Experience I'm learning it and occasionally try some simple examples I'm currently developing applications based on LLMs. The application is not yet in production. I've developed applications based on LLMs. The application is already in production. Question Title * 7. What do you think is most challenging when writing prompts for LLM I completely unsure where to start. Sometimes the results are correct, sometimes not, and I don't know how to adjust The results seem correct, but I'm unsure about their effectiveness in real user scenarios. I'm proficient with basic methods, but still don't understand advanced techniques such as CoT, ToT, etc. I'm a prompt expert, there are no more challenges for me. Other (please specify) Question Title * 8. How do you usually test/optimize your prompts? Manually test a few cases, as long as it works. Manually test many cases to cover as many scenarios as possible. Maintain a test set, run batch tests after each adjustment. I am not responsible for testing. Question Title * 9. Your Opinion on the Token Costs Incurred During Prompt Tuning Completely unacceptable Acceptable if the cost is minimal, no more than 1-2 USD each time Acceptable as long as it's within 10 USD each time Insensitive to cost, saving my time is most important Insensitive to cost, improving product quality is most important Question Title * 10. If there is an Automated Prompt Optimization Plugin, do you think it's useful? Useless Useful, I would use it occasionally Very useful, I would definitely use it Question Title * 11. If there is an Automated Prompt Optimization Plugin, are you willing to pay an annual fee for using it? Yes No Question Title * 12. Your comments (optional) Done