As a quick sanity check to make sure the bot files and localization changes are imported properly to the dev server, one can check https://covid19healthbot-dev.cdc.gov/ and run through a quick path.  This will be done more thoroughly in log/data testing.


Below are a few suggestions to help with testing the bot's code and doing front-end testing. A big key to doing this type of code testing is to have the developer be neat and consistent with their coding. If the code is very messy or the developer is not consistent, this process may not work.

Before beginning, make sure to have a text editor that allows line sorting and a mode that allows the selection of text in columns. 

    1. For Notepad++
      1. Line sorting is under Edit → Line Operations → Sort Line Lexicographically Ascending
      2. Column Mode is under Edit → Column Mode or hold down the Alt key as you select text with the mouse.

All testing should be done in a non-production environment. Bots should not be edited directly in production. 


Checking for correct messaging

  1. What is being done here:
    1. Making sure the correct messages are given.
    2. Making sure messages are given in the correct order.
    3. Making sure Action names are correct
  2. Steps:
    1. Open bot in Azure Health Bot editor
    2. Go to Code view
    3. Click in code area and hit Ctrl-A to select all of the code
    4. Hit Ctrl-C to copy the code
    5. Open a new file in the text editor and hit Ctrl-P to paste the code
    6. Sort the lines of code. This will group similar lines of code together.
    7. Scroll down to the lines that begin with "label".
    8. Here you will find lines that include the Message IDs.
    9. These are the labels for the Action items.
    10. Delete all the lines above these lines and any lines in between the lines we care about.
    11. Scroll down to the lines that begin with "onInit".
    12. Here you will find lines that contain the Message IDs.
    13. These contain the code found inside of the action items.
    14. Delete all the lines below these lines.
    15. Delete all the lines between the "label" lines you care about and the "onInit" lines you care about.
    16. What is left should look similar to this:
    17. Using the Column Mode feature, remove the ""label": "" and the ""onInit": "scenario.outcomeID = '" text from the front of these lines.
    18. What is left should look like this:
    19. Sort the lines of code.
    20. What is left should look like this:
    21. Note that the test from the "label" lines are now next to the text of the corresponding "onInit" lines. 
    22. Now review the code to make sure the Message IDs match, the label text is correct, the correct messages are given and the messages are given in the correct order. 
    23. Tips:
      1. Compare the message given and their order to the bot's documentation. As a line is verified, either mark it some way in the text editor or delete the line from the text editor document.

      2. All of this should be done inside a text editor.

        Do not do any editing inside the Code viewer of the Azure HealthBot app!!!

Checking for correct Outcome code

  1. What is being done here:
    1. Comparing Outcome code to make sure all variables are being used
    2. Comparing Outcome code to make sure the code matches correctly (for those blocks that should match)
    3. Looking for anything that looks off in the code
  2. Steps:
    1. Open bot in Azure Health Bot editor
    2. Go to Code view
    3. Click in code area and hit Ctrl-A to select all of the code
    4. Hit Ctrl-C to copy the code
    5. Open a new file in the text editor and hit Ctrl-P to paste the code
    6. Do this for all bots that are part of the same project. All code goes into separate documents in the text editor. 
    7. Sort the lines of code in each document. This will group similar lines of code together.
    8. Look for the Outcome code. This will be in the area of the code that begins with "onInit". You may need to look at the code in the Action items in the bot diagram to see what you are looking for. Depending on how the bot is written, there could be multiple Outcome code lines. 
    9. Once found, delete all lines of code except for these lines.
    10. Do this for each document in the text editor containing code.
    11. The next step is to combine all of these lines into a single document. Before doing this make sure to add a marker at the beginning of each line to identify which bot code it came from. This is very important otherwise once all the code is in a single document, it will be hard to find which bot it came from.
    12. Once the lines of code are combined in a single document, it should look similar to this:
    13. If the developer was neat and consistent, hopefully the code will line up. Review the code and look for anything that seems out of place. Note how in the below image the alignment is off. In this case this is pointing to an error in that misaligned code line.

Visually reviewing bot via Designer view

  1. What is being done here:
    1. Making sure the visual representation of the bot looks good
    2. Looking for anything that looks out of place.
  2. Steps
    1. Go to the Designer view
    2. This process is more "look around and see what looks off". Being familiar with the bot logic and how the bot looks in the Designer view will greatly help.
    3. Click the Run button at walk through the bot watching how each part of the bot flows.
    4. Answer questions using the Web Chat box
    5. If the bot is small enough, use the Watch window to follow the bots progress. Items here can be expanded to see values of parameters as they are selected. i.e. In the image below, "Male" was selected for question showing in Web Chat and the selections shows as the value for the variable "gender" in the Watch window.

      1. If the bot is too large and the Watch window shows the error "Data is too large to be shown in the debugger", there is a work-around. The areas of the bot not being tested can be deleted. Doing this can make the bot small enough to test specific areas. 

        This should only be done in a non-production environment and using a copy that will not cause problems if lost. 

    6. If the bot is small enough, use the Immediate window to find the value of a specific variable. i.e. In the image below, typing "scenario.gender" and hitting the Evaluate button returns the current value of the "gender" variable.

Checking the pathways

  1. What is being done here:
    1. Walking through all the different pathway using the front-end
    2. Checking for correct flow of questions
    3. Checking for correct messaging
    4. Checking that questions with checkboxes work as expected
  2. Steps
    1. Open a non-production version of the bot in a browser
    2. Walk through the bot and do the following:
      1. answer questions and check for correct flow
      2. check and uncheck boxes making sure they work as expected
      3. look at questions to make sure they are formatted correctly
      4. look at the message returned at the end making sure they are correct and formatted correctly
    3. Use the bot documentation to track your progress. If any issues are found, provide the question number or message number or message ID or screen shot to the developer so they know exactly where the issue is located. 
    4. Check the database connected to the bot to verify the data being collected it correct. If the review of the backend code went well, hopefully there are no issues here. 

Checking the Localization data

  1. What is being done here:
    1. Looking at all the text used in the bot to make sure it is correct
    2. Checking for typos, formatting issues, Markdown formatting issues, syntax
  2. Steps
    1. Get latest copy of Localization file from developer. It will be an Excel file
    2. Review the text to make sure it matches the bot documentation
    3. Check the Markdown formatting to see how it will look inside the bot. Some free sites for viewing Markdown are:
      1. Best Markdown Viewer Online and Markdown Tester Online (codebeautify.org)
      2. Markdown Live Preview
    4. Make corrections inside the Excel file
    5. Once complete, send the Excel file back to the developer so it can load back into the Azure Healthbot app. Check with the developer as they may only what changes in the file or they may be OK with receiving the whole file.
  • No labels