Distest

Distest makes it easy to write application tests for discord bots.

Distest uses a secondary bot to send commands to your bot and ensure that it responds as expected.

See the interface reference for a list of assertions this library is capable of.

Note

Two quick note about recent changes:

  1. You NEED to enable the members intent on the tester bot. For more information, see Member Intent
  2. If you’re using the ext.commands.Bot system, you will need to patch your Bot to allow it to listen to other discord bots, as usually commands ignore other bots. This is really easy, we provide the patching function, just take a look at the patching documentation page.

Quickstart

Installation

  1. Install the library with pip:
    $ pip install distest

  2. Distest works by using a second bot (the ‘tester’) to assert that your bot (the ‘target’) reacts to events appropriately. This means you will need to create a second bot account through the Discord Developer’s Portal and obtain the authorization token. You also have to invite the tester to your discord guild. Additionally, be sure to enable the SERVER MEMBERS INTENT option, see Member Intent docs for more info.

  3. Refer to the Example Test Suite for the syntax/function calls necessary to build your suite.

Usage

The tests can be run in one of two modes: interactive and command-line. In interactive mode, the bot will wait for you to initiate tests manually. In command-line mode, the bot will join a designated channel, run all designated tests, and exit with a code of 0 if all tests were successful and any other number if the one or more tests failed. This allows for automating your test suite, allowing you to implement Continuous Integration on your Discord bot!

No matter how you run your tester, the file must contain:

  1. A call to run_dtest_bot, which will handle all command line arguments and run the tester in the correct mode
  2. A TestCollector, which will let the bot find and run the you specify
  3. One or more Test, which should be decorated with the TestCollector, and are the actual tests that are run.

Note

The error codes will currently be 0 on success or 1 on failure, but we plan to implement meaningful error codes

Interactive Mode

  1. Run the bot by running your test suite module directly (called example_tester.py here):
    $ python example_tester.py TARGET_ID TESTER_TOKEN

  2. Go to the channel you want to run your tests in and call the bot using the ::run command. You can either designate specific tests to run by name or use ::run all

See also

::help command for more commands/options.

Command-Line Mode

For command-line you have to designate the ID of the channel you want to run tests in (preceded by the -c flag). You must also designate which tests to run (with the -r flag). Your command should look something like this:

$ python example_tester.py TARGET_ID TESTER_TOKEN -c CHANNEL_ID -r all

The program will print test names to the console as it runs them, and then exit.

See also

readme.md on GitHub, which contains a more in-depth look at the command properties

Example Test Suite

This is the example_tester.py file found in the root directory. It contains tests for every assertion in Interface. This suite is also used to test our library, in conjunction with the example_target.py. The easiest way to get started is to adapt this suite of tests so it’s specific to your bot, then run this module with

$ python example_tester.py ${TARGET_ID} ${TESTER_TOKEN}

where TARGET_ID is the Discord ID of your bot, and TESTER_TOKEN is the auth token for your testing bot.

  1
  2
  3
  4
  5
  6
  7
  8
  9
 10
 11
 12
 13
 14
 15
 16
 17
 18
 19
 20
 21
 22
 23
 24
 25
 26
 27
 28
 29
 30
 31
 32
 33
 34
 35
 36
 37
 38
 39
 40
 41
 42
 43
 44
 45
 46
 47
 48
 49
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
"""
A functional demo of all possible test cases. This is the format you will want to use with your testing bot.

    Run with:
        python example_tests.py TARGET_NAME TESTER_TOKEN
"""
import asyncio
import sys
from distest import TestCollector
from distest import run_dtest_bot
from discord import Embed, Member, Status
from distest import TestInterface

# The tests themselves

test_collector = TestCollector()
created_channel = None


@test_collector()
async def test_ping(interface):
    await interface.assert_reply_contains("ping?", "pong!")


@test_collector()
async def test_delayed_reply(interface):
    message = await interface.send_message(
        "Say some stuff, but at 4 seconds, say 'yeet'"
    )
    await interface.get_delayed_reply(5, interface.assert_message_equals, "yeet")


@test_collector()
async def test_reaction(interface):
    await interface.assert_reaction_equals("React with \u2714 please!", u"\u2714")


@test_collector()
async def test_reply_equals(interface):
    await interface.assert_reply_equals("Please say 'epic!'", "epic!")


@test_collector()
async def test_channel_create(interface):
    await interface.send_message("Create a tc called yeet")
    created_channel = await interface.assert_guild_channel_created("yeet")


# @test_collector
# async def test_pin_in_channel(interface):
#     await interface.send_message("Pin 'this is cool' in yeet")
#     await interface.assert_guild_channel_pin_content_equals(created_channel )


@test_collector()
async def test_channel_delete(interface):
    await interface.send_message("Delete that TC bro!")
    await interface.assert_guild_channel_deleted("yeet")


@test_collector()
async def test_silence(interface):
    await interface.send_message("Shhhhh...")
    await interface.ensure_silence()


@test_collector()
async def test_reply_contains(interface):
    await interface.assert_reply_contains(
        "Say something containing 'gamer' please!", "gamer"
    )


@test_collector()
async def test_reply_matches(interface):
    await interface.assert_reply_matches(
        "Say something matching the regex `[0-9]{1,3}`", r"[0-9]{1,3}"
    )


@test_collector()
async def test_ask_human(interface):
    await interface.ask_human("Click the Check!")


@test_collector()
async def test_embed_matches(interface):
    embed = (
        Embed(
            title="This is a test!",
            description="Descriptive",
            url="http://www.example.com",
            color=0x00FFCC,
        )
            .set_author(name="Author")
            .set_thumbnail(
            url="https://upload.wikimedia.org/wikipedia/commons/4/40/Test_Example_%28cropped%29.jpg"
        )
            .set_image(
            url="https://upload.wikimedia.org/wikipedia/commons/4/40/Test_Example_%28cropped%29.jpg"
        )
    )

    # This image is in WikiMedia Public Domain
    await interface.assert_reply_embed_equals("Test the Embed!", embed)


@test_collector()
async def test_embed_part_matches(interface):
    embed = Embed(title="Testing Title.", description="Wrong Description")
    await interface.assert_reply_embed_equals(
        "Test the Part Embed!", embed, attributes_to_check=["title"]
    )


@test_collector()
async def test_reply_has_image(interface):
    await interface.assert_reply_has_image("Post something with an image!")


@test_collector()
async def test_reply_on_edit(interface):
    message = await interface.send_message("Say 'Yeah, that cool!'")
    await asyncio.sleep(1)
    await interface.edit_message(message, "Say 'Yeah, that is cool!'")
    await interface.assert_message_contains(message, "Yeah, that is cool!")


@test_collector()
async def test_send_message_in_channel(interface):
    message = await interface.send_message("Say stuff in another channel")
    await interface.wait_for_message_in_channel("here is a message in another channel", 694397509958893640)


# Actually run the bot

if __name__ == "__main__":
    run_dtest_bot(sys.argv, test_collector)

The Member Intent

Discord recently changed what you have to do for bots to be able to access server member informaiton, meaning that without changes, calling guild.members will return an empty list, which is no good!!

To fix this, we need to do two things:

  1. Enable the Privileged Gateway Intent for Server Members.
    1. To do this. go to the Discord developer portal and select your tester bot
    2. Select the bot tab
    3. Enable the SERVER MEMBERS INTENT and the PRESENCE INTENT` sliders
  2. Update distest! There are also changes that need to be made on our side. They have been made, but be sure you update to 0.4.9 or newer to get the changes!

Now, you should be good to go. Have fun testing!

Quick note - For some godforsaken reason, the on_member_update event is just horribly slow and unreliable. I’m not really sure what to do about this, but be forewarned if you want to use it!

Main Functions

Interface

This is the most important class in the library for you, as it contains all the assertions and tools you need to interface with the library. Generally broken down into a few overall types:

  • Message (i.e. assert_message_contains): Does not send it’s own message, so it require a Message to be passed in.
  • Reply (i.e. assert_reply_contains): Sends a message containing the text in contents and analyzes messages sent after that.
    • Use get_delayed_reply to wait an amount of time before checking for a reply
  • Embed (i.e. assert_embed_equals): Sends a message then checks the embed of the response against a list of attributes
  • Other Tests (i.e. ask_human): Some tests do weird things and don’t have a clear category.
  • Interface Functions (i.e. connect, send_message): Help other tests but also can be useful in making custom tests out of the other tests.

Enumerations

The following enumeration (subclass of enum.Enum) is used to indicate the result of a run test.

class TestResult

Specifies the result of a test.

UNRUN

Test has not been run in this session

SUCCESS

Test succeeded

FAILED

Test has failed.

Bot

Collector

Exceptions

Patches

Contains the code required to patch out the fact that Bot class ignores messages from other bots.

This should be used if you have a target bot that uses the ext.commands.Bot system, as otherwise it’s commands will ignore messages from the tester bot.

Usage

Simply put the below code into your main bot and then when testing, the bot will no longer ignore other bots!

1
2
3
4
5
6
bot = commands.Bot(command_prefix='$')

# Do anything you want for this if, be it env vars, command line args, or the likes.
if sys.argv[2] == "TESTING":
    from distest.patches import patch_target
    bot = patch_target(bot)

Docs

Contributing

Hey! You’re here because you want to contribute to the bot, and that’s awesome! Here are some notes about contributions:

  • Please open an issue for your contribution and tag it with contribution to discuss it. Because of my time constraints, I probably won’t have much time to implement new features myself, but if you make a feature and PR it in, I’ll be more than happy to spend a bit of time testing it out and add it to the project. The other thing is to make sure you check the github project to see if there is someone else already working on it who you can help.
  • You may need to install the additional requirements from requirements-dev.txt. This is as simple as running pip install -r requirements-dev.txt. This larger list mostly includes things like black for formatting and sphinx for doc testing.
  • If you are adding new test types, please make sure you test them well to make sure they work as intended, and please add a demo of them in use to the example_tests() for others to see. When you are done, please open a PR and I’ll add it in!
  • I use Black for my code formatting, it would be appreciated if you could use it when contributing as well. I will remind you when you make a PR if you don’t, it is essential to make sure that diffs aren’t cluttered up with silly formatting changes. Additionally, CodeFactor should be tracking code quality and doing something with PRs. We will see soon exactly how that will work out.
  • To build the docs for testing purposes, cd into the docs folder and run make testhtml. This will build to a set of HTML files you can view locally.
  • Also, if you just want to propose an idea, create an issue and tag it with enhancement. The library is missing tons of features, so let me know what you want to see, and if I have time I’ll see about getting around to addign it. Thank you for your help!

Meta Documentation Pages