What I learned about test automation

I’ve overcome the first major leap of test automation: Management is behind me and has agreed to fund it by providing me the complete training with drills that carefully designed to be benefited in our industry. Now what? I've got plenty of test strategies, approaches and tactics. But where do I start? My experience has shown that just because a test is automatable doesn't necessary mean it should be automated. Therefore, analysis takes into account limited resources, budgets, schedules, and available expertise.

Test Automation Criteria
1. Is the test executed more than once?
2. Is the test run on a regular basis, i.e., reuse potential is high, such as part of regression or build testing?
3. Does the test cover most critical feature paths?
4. Does the test cover high risk areas?
5. Is the test impossible or prohibitively expensive to perform manually, such as concurrency, soak/endurance testing, performance, and memory leak detection testing?
6. Are there timing-critical components and dependencies that are a must to automate?
7. Does the test cover the most complex area (often the most error-prone)?
8. Does the test require many data combination using the same test steps (i.e., multiple data inputs for the same feature)?
9. Are the expected results constant, i.e., do not change or vary with each test? Even if the results vary, is there a percentage tolerance that could be measured as expected results?
10. Is the test outcome analysis overly time-consuming, such as expected results analysis of hundreds of outputs?
11. Does the test need to be verified on multiple software and hardware configurations?
12. Does the test automation ROI look promising and meet any organizational ROI criteria?

Sample checklist for deciding what to automate

Cases for which the answer is "yes" are good candidates for automation, provided your organizational resources and expertise allow