[Python] え、まだtox使ってないんですか?

今回はtoxです。さんざん記事書くって言ってあったので書きます(本当は4月の予定だったけどね
この記事は神々(@aodag, @shimzukawa)の知識をほとんどそのまま拝借したものです。ほぼ乗っ取られてます。

準備

せっかくtoxを使うので複数バージョンのpythonでテストしたいんですが、私はanacondaやpyenvが苦手なので、開発環境を作りました。
もちろん無理に使う必要はないですし、自分の環境でやるという方はそちらのほうがよいです。
(ちょっと脱線して利用方法を説明

  1. $ git clone git@github.com:righ/devenv.git
  2. $ cd devenv/
  3. $ git checkout full-python
  4. $ vagrant up (失敗した場合はvagrant provision)
  5. (プロビジョンが自動的に実行されなかった場合は)$ ansible-provision -i inventory/vagrant provision.yml # もしくはvagrant provision
  6. $ vagrant ssh

これが成功すると python2.5 ~ 3.5 のpythonがインストールされたVMができます。全然fullじゃねーじゃん

[vagrant@localhost ~]$ python
python             python2.6-config   python3.1          python3.3          python3.4m-config
python2            python2.7          python3.1-config   python3.3m         python3.5
python2.5          python2.7-config   python3.2          python3.3m-config  python3.5m
python2.5-config   python3.0          python3.2m         python3.4          python3.5m-config
python2.6          python3.0-config   python3.2m-config  python3.4m

[vagrant@localhost ~]$ pip
pip     pip2    pip2.6  pip2.7  pip3    pip3.3  pip3.4  pip3.5

(脱線終わり)
今回テストの対象とするのは私が公開してるtesdatというライブラリを使います。
もちろん試したいコードがある方はそちらでよいです。
VMでcloneします。

[vagrant@localhost ~]$ cd projects/ # projects配下はホスト間と共有されてる
[vagrant@localhost projects]$ git clone https://github.com/righ/tesdat
[vagrant@localhost projects]$ cd tesdat/

準備はこれで終了。

導入

ではインストール。このときvenvを有効にしてもしなくてもOKです。私はやりますが。

$ pip install tox

残念ながらまだtoxは使えません。

(venv3.5) [vagrant@localhost tesdat]$ tox
ERROR: toxini file 'tox.ini' not found

toxを使うにはtox.iniという設定ファイルが必要です。

ひとまずこんな風にしましょう。

[tox]
envlist = py27, py33, py34, py35, flake8
 
[testenv]
commands = py.test
deps =
  pytest
 
[testenv:flake8]
deps =
    flake8
    flake8-blind-except
    flake8-docstrings
    flake8-import-order
 
commands = flake8 .
 
[flake8]
exclude = tests/*

簡単に説明すると

  • テストランナーはpy.testを使う
  • python2.7, python3.3, python3.4, python3.5でテストを実施する。python3.2(というかpip3.2?)がtoxでうまく動作しなかったので3.3からにしました。しょうがないね。
  • tests/フォルダを除いてflake8を実施する。
  • flake8での構文を確認するためにライブラリをインストールする

って感じですね。
tox.iniで特に重要なのは「tox」と「testenv」セクションです。
「tox」セクションの「envlist」に記述したコマンドが実際に何をするかやオプションを「testenv:~」に記述します。
デフォルトenvは「testenv」セクションに属します。簡単に言うと「py*(python*)」「jython」「pypy」です。詳しくはこちらを参照。
特定の環境だけでテストを実行したい場合は「-e」オプションを使います。例えば上記設定を用いてflake8だけを実行したい場合は「tox -e flake8」のようにします。

depsにはテストに必要なライブラリをrequirements.txt形式で指定できます。
すでにプロジェクトでrequirements.txtがある場合「-rrequirements.txt」のように指定できます。テスト用のrequirements.txtを作っておくのがよいでしょう。

とりあえず実行してみましょう。

(venv3.5) [vagrant@localhost tesdat]$ tox
GLOB sdist-make: /home/vagrant/tesdat/setup.py
py27 inst-nodeps: /home/vagrant/tesdat/.tox/dist/tesdat-2.1.1.zip
py27 installed: tesdat==2.1.1
py27 runtests: PYTHONHASHSEED='1127406230'
py27 runtests: commands[0] | py.test
WARNING:test command found but not installed in testenv
  cmd: /home/vagrant/venv3.5/bin/py.test
  env: /home/vagrant/tesdat/.tox/py27
Maybe you forgot to specify a dependency? See also the whitelist_externals envconfig setting.
WARNING: Attempting to work in a virtualenv. If you encounter problems, please install IPython inside the virtualenv.
WARNING: IPython History requires SQLite, your history will not be saved
============================================== test session starts ===============================================
platform linux -- Python 3.5.1, pytest-2.9.1, py-1.4.31, pluggy-0.3.1
rootdir: /home/vagrant/tesdat, inifile:
plugins: ipdb-0.1.dev2, django-2.9.1
collected 57 items

tesdat/tests/test_api.py ....
tesdat/tests/test_example.py ...
tesdat/tests/test_index.py ..
tesdat/tests/containers/test_base.py ...
tesdat/tests/containers/test_dict.py ...
tesdat/tests/containers/test_iter.py ...
tesdat/tests/containers/test_list.py ..
tesdat/tests/containers/test_ordereddict.py .
tesdat/tests/formatters/test_csv.py ...
tesdat/tests/formatters/test_json.py F
tesdat/tests/formatters/test_pickle.py .
tesdat/tests/formatters/test_string.py F.
tesdat/tests/models/test_dict.py .....
tesdat/tests/models/test_list.py ......
tesdat/tests/patterns/test_choice.py ..
tesdat/tests/patterns/test_cycle.py ..
tesdat/tests/patterns/test_hashof.py .
tesdat/tests/patterns/test_increment.py ..
tesdat/tests/patterns/test_pickout.py ..
tesdat/tests/patterns/test_sequence.py ..
tesdat/tests/utils/test_datetime.py .......

==================================================== FAILURES ====================================================
________________________________________ TestJsonFormatter.test_stringify ________________________________________

self = 

    def test_stringify(self):
        sf = self._getClass({'a': time(23, 59), 'b': ('2',)}, indent=None)
        self.assertEqual(
            sf.stringify(),
>           '{"a": "23:59:00", "b": ["2"]}'
        )
E       AssertionError: '{"b": ["2"], "a": "23:59:00"}' != '{"a": "23:59:00", "b": ["2"]}'
E       - {"b": ["2"], "a": "23:59:00"}
E       + {"a": "23:59:00", "b": ["2"]}

tesdat/tests/formatters/test_json.py:15: AssertionError
________________________________________ TestStringFormatter.test_string _________________________________________

self = 

    def test_string(self):
        sf = self._getClass({'a': 1, 'b': ('2',)})
        self.assertEqual(
            sf.stringify(),
>           "{'a': 1, 'b': ('2',)}"
        )
E       AssertionError: "{'b': ('2',), 'a': 1}" != "{'a': 1, 'b': ('2',)}"
E       - {'b': ('2',), 'a': 1}
E       + {'a': 1, 'b': ('2',)}

tesdat/tests/formatters/test_string.py:23: AssertionError
====================================== 2 failed, 55 passed in 0.95 seconds =======================================
ERROR: InvocationError: '/home/vagrant/venv3.5/bin/py.test'
py33 inst-nodeps: /home/vagrant/tesdat/.tox/dist/tesdat-2.1.1.zip
py33 installed: tesdat==2.1.1
y33 runtests: PYTHONHASHSEED='1127406230'
py33 runtests: commands[0] | py.test
WARNING:test command found but not installed in testenv
  cmd: /home/vagrant/venv3.5/bin/py.test
  env: /home/vagrant/tesdat/.tox/py33
Maybe you forgot to specify a dependency? See also the whitelist_externals envconfig setting.
WARNING: Attempting to work in a virtualenv. If you encounter problems, please install IPython inside the virtualenv.
WARNING: IPython History requires SQLite, your history will not be saved
============================================== test session starts ===============================================
platform linux -- Python 3.5.1, pytest-2.9.1, py-1.4.31, pluggy-0.3.1
rootdir: /home/vagrant/tesdat, inifile:
plugins: ipdb-0.1.dev2, django-2.9.1
collected 57 items

tesdat/tests/test_api.py ....
tesdat/tests/test_example.py ...
tesdat/tests/test_index.py ..
tesdat/tests/containers/test_base.py ...
tesdat/tests/containers/test_dict.py ...
tesdat/tests/containers/test_iter.py ...
tesdat/tests/containers/test_list.py ..
tesdat/tests/containers/test_ordereddict.py .
tesdat/tests/formatters/test_csv.py ...
tesdat/tests/formatters/test_json.py F
tesdat/tests/formatters/test_pickle.py .
tesdat/tests/formatters/test_string.py F.
tesdat/tests/models/test_dict.py .....
tesdat/tests/models/test_list.py ......
tesdat/tests/patterns/test_choice.py ..
tesdat/tests/patterns/test_cycle.py ..
tesdat/tests/patterns/test_hashof.py .
tesdat/tests/patterns/test_increment.py ..
tesdat/tests/patterns/test_pickout.py ..
tesdat/tests/patterns/test_sequence.py ..
tesdat/tests/utils/test_datetime.py .......

==================================================== FAILURES ====================================================
________________________________________ TestJsonFormatter.test_stringify ________________________________________

self = 

    def test_stringify(self):
        sf = self._getClass({'a': time(23, 59), 'b': ('2',)}, indent=None)
        self.assertEqual(
            sf.stringify(),
>           '{"a": "23:59:00", "b": ["2"]}'
        )
E       AssertionError: '{"b": ["2"], "a": "23:59:00"}' != '{"a": "23:59:00", "b": ["2"]}'
E       - {"b": ["2"], "a": "23:59:00"}
E       + {"a": "23:59:00", "b": ["2"]}

tesdat/tests/formatters/test_json.py:15: AssertionError
________________________________________ TestStringFormatter.test_string _________________________________________

self = 

    def test_string(self):
        sf = self._getClass({'a': 1, 'b': ('2',)})
        self.assertEqual(
            sf.stringify(),
>           "{'a': 1, 'b': ('2',)}"
        )
E       AssertionError: "{'b': ('2',), 'a': 1}" != "{'a': 1, 'b': ('2',)}"
E       - {'b': ('2',), 'a': 1}
E       + {'a': 1, 'b': ('2',)}

tesdat/tests/formatters/test_string.py:23: AssertionError
====================================== 2 failed, 55 passed in 0.93 seconds =======================================
ERROR: InvocationError: '/home/vagrant/venv3.5/bin/py.test'
py34 inst-nodeps: /home/vagrant/tesdat/.tox/dist/tesdat-2.1.1.zip
py34 installed: tesdat==2.1.1
py34 runtests: PYTHONHASHSEED='1127406230'
py34 runtests: commands[0] | py.test
WARNING:test command found but not installed in testenv
  cmd: /home/vagrant/venv3.5/bin/py.test
  env: /home/vagrant/tesdat/.tox/py34
Maybe you forgot to specify a dependency? See also the whitelist_externals envconfig setting.
WARNING: Attempting to work in a virtualenv. If you encounter problems, please install IPython inside the virtualenv.
WARNING: IPython History requires SQLite, your history will not be saved
============================================== test session starts ===============================================
platform linux -- Python 3.5.1, pytest-2.9.1, py-1.4.31, pluggy-0.3.1
rootdir: /home/vagrant/tesdat, inifile:
plugins: ipdb-0.1.dev2, django-2.9.1
collected 57 items

tesdat/tests/test_api.py ....
tesdat/tests/test_example.py ...
tesdat/tests/test_index.py ..
tesdat/tests/containers/test_base.py ...
tesdat/tests/containers/test_dict.py ...
tesdat/tests/containers/test_iter.py ...
tesdat/tests/containers/test_list.py ..
tesdat/tests/containers/test_ordereddict.py .
tesdat/tests/formatters/test_csv.py ...
tesdat/tests/formatters/test_json.py F
tesdat/tests/formatters/test_pickle.py .
tesdat/tests/formatters/test_string.py F.
tesdat/tests/models/test_dict.py .....
tesdat/tests/models/test_list.py ......
tesdat/tests/patterns/test_choice.py ..
tesdat/tests/patterns/test_cycle.py ..
tesdat/tests/patterns/test_hashof.py .
tesdat/tests/patterns/test_increment.py ..
tesdat/tests/patterns/test_pickout.py ..
tesdat/tests/patterns/test_sequence.py ..
tesdat/tests/utils/test_datetime.py .......

==================================================== FAILURES ====================================================
________________________________________ TestJsonFormatter.test_stringify ________________________________________

self = 

    def test_stringify(self):
        sf = self._getClass({'a': time(23, 59), 'b': ('2',)}, indent=None)
        self.assertEqual(
            sf.stringify(),
>           '{"a": "23:59:00", "b": ["2"]}'
        )
E       AssertionError: '{"b": ["2"], "a": "23:59:00"}' != '{"a": "23:59:00", "b": ["2"]}'
E       - {"b": ["2"], "a": "23:59:00"}
E       + {"a": "23:59:00", "b": ["2"]}

tesdat/tests/formatters/test_json.py:15: AssertionError
________________________________________ TestStringFormatter.test_string _________________________________________

self = 

    def test_string(self):
        sf = self._getClass({'a': 1, 'b': ('2',)})
        self.assertEqual(
            sf.stringify(),
>           "{'a': 1, 'b': ('2',)}"
        )
E       AssertionError: "{'b': ('2',), 'a': 1}" != "{'a': 1, 'b': ('2',)}"
E       - {'b': ('2',), 'a': 1}
E       + {'a': 1, 'b': ('2',)}

tesdat/tests/formatters/test_string.py:23: AssertionError
====================================== 2 failed, 55 passed in 0.92 seconds =======================================
ERROR: InvocationError: '/home/vagrant/venv3.5/bin/py.test'
py35 inst-nodeps: /home/vagrant/tesdat/.tox/dist/tesdat-2.1.1.zip
py35 installed: tesdat==2.1.1
py35 runtests: PYTHONHASHSEED='1127406230'
py35 runtests: commands[0] | py.test
WARNING:test command found but not installed in testenv
  cmd: /home/vagrant/venv3.5/bin/py.test
  env: /home/vagrant/tesdat/.tox/py35
Maybe you forgot to specify a dependency? See also the whitelist_externals envconfig setting.
WARNING: Attempting to work in a virtualenv. If you encounter problems, please install IPython inside the virtualenv.
WARNING: IPython History requires SQLite, your history will not be saved
============================================== test session starts ===============================================
platform linux -- Python 3.5.1, pytest-2.9.1, py-1.4.31, pluggy-0.3.1
rootdir: /home/vagrant/tesdat, inifile:
plugins: ipdb-0.1.dev2, django-2.9.1
collected 57 items

tesdat/tests/test_api.py ....
tesdat/tests/test_example.py ...
tesdat/tests/test_index.py ..
tesdat/tests/containers/test_base.py ...
tesdat/tests/containers/test_dict.py ...
tesdat/tests/containers/test_iter.py ...
tesdat/tests/containers/test_list.py ..
tesdat/tests/containers/test_ordereddict.py .
tesdat/tests/formatters/test_csv.py ...
tesdat/tests/formatters/test_json.py F
tesdat/tests/formatters/test_pickle.py .
tesdat/tests/formatters/test_string.py F.
tesdat/tests/models/test_dict.py .....
tesdat/tests/models/test_list.py ......
tesdat/tests/patterns/test_choice.py ..
tesdat/tests/patterns/test_cycle.py ..
tesdat/tests/patterns/test_hashof.py .
tesdat/tests/patterns/test_increment.py ..
tesdat/tests/patterns/test_pickout.py ..
tesdat/tests/patterns/test_sequence.py ..
tesdat/tests/utils/test_datetime.py .......

==================================================== FAILURES ====================================================
________________________________________ TestJsonFormatter.test_stringify ________________________________________

self = 

    def test_stringify(self):
        sf = self._getClass({'a': time(23, 59), 'b': ('2',)}, indent=None)
        self.assertEqual(
            sf.stringify(),
>           '{"a": "23:59:00", "b": ["2"]}'
        )
E       AssertionError: '{"b": ["2"], "a": "23:59:00"}' != '{"a": "23:59:00", "b": ["2"]}'
E       - {"b": ["2"], "a": "23:59:00"}
E       + {"a": "23:59:00", "b": ["2"]}

tesdat/tests/formatters/test_json.py:15: AssertionError
________________________________________ TestStringFormatter.test_string _________________________________________

self = 

    def test_string(self):
        sf = self._getClass({'a': 1, 'b': ('2',)})
        self.assertEqual(
            sf.stringify(),
>           "{'a': 1, 'b': ('2',)}"
        )
E       AssertionError: "{'b': ('2',), 'a': 1}" != "{'a': 1, 'b': ('2',)}"
E       - {'b': ('2',), 'a': 1}
E       + {'a': 1, 'b': ('2',)}

tesdat/tests/formatters/test_string.py:23: AssertionError
====================================== 2 failed, 55 passed in 0.99 seconds =======================================
ERROR: InvocationError: '/home/vagrant/venv3.5/bin/py.test'
flake8 recreate: /home/vagrant/tesdat/.tox/flake8
flake8 installdeps: flake8, flake8-blind-except, flake8-docstrings, flake8-import-order, mccabe, radon
flake8 inst: /home/vagrant/tesdat/.tox/dist/tesdat-2.1.1.zip
flake8 installed: colorama==0.3.7,flake8==2.5.4,flake8-blind-except==0.1.0,flake8-docstrings==0.2.5,flake8-import-order==0.7,mando==0.3.3,mccabe==0.4.0,pep257==0.7.0,pep8==1.7.0,pyflakes==1.0.0,radon==1.3.2,tesdat==2.1.1
flake8 runtests: PYTHONHASHSEED='1127406230'
flake8 runtests: commands[0] | flake8 .
./setup.py:1:1: D100 Missing docstring in public module
./setup.py:4:1: I101 Imported names are in the wrong order. Should be find_packages, setup
./setup.py:5:1: I101 Imported names are in the wrong order. Should be __author__, __author_email__, __license__, __version__
./setup.py:5:1: I201 Missing newline before sections or imports.
./tesdat/__init__.py:1:1: D200 One-line docstring should fit on one line with quotes
./tesdat/__init__.py:11:1: E402 module level import not at top of file
./tesdat/__init__.py:12:1: E402 module level import not at top of file
./tesdat/__init__.py:12:1: I100 Import statements are in the wrong order. from .containers.dict should be before from .containers.list
./tesdat/__init__.py:13:1: E402 module level import not at top of file
./tesdat/__init__.py:15:1: E402 module level import not at top of file
./tesdat/__init__.py:16:1: E402 module level import not at top of file
./tesdat/__init__.py:16:1: I100 Import statements are in the wrong order. from .models.dict should be before from .models.list
./tesdat/__init__.py:18:1: E402 module level import not at top of file
./tesdat/__init__.py:19:1: E402 module level import not at top of file
./tesdat/__init__.py:20:1: E402 module level import not at top of file
./tesdat/__init__.py:20:1: I100 Import statements are in the wrong order. from .patterns.cycle should be before from .patterns.pickout
./tesdat/__init__.py:21:1: E402 module level import not at top of file
./tesdat/__init__.py:22:1: E402 module level import not at top of file
./tesdat/__init__.py:23:1: E402 module level import not at top of file
./tesdat/__init__.py:26:1: E402 module level import not at top of file
./tesdat/__init__.py:26:1: I100 Import statements are in the wrong order. from .formatters.string should be before from .patterns.sequence
./tesdat/__init__.py:27:1: E402 module level import not at top of file
./tesdat/__init__.py:27:1: I100 Import statements are in the wrong order. from .formatters.json should be before from .formatters.string
./tesdat/__init__.py:28:1: E402 module level import not at top of file
./tesdat/__init__.py:29:1: E402 module level import not at top of file
./tesdat/__init__.py:29:1: I100 Import statements are in the wrong order. from .formatters.csv should be before from .formatters.pickle
./tesdat/__init__.py:31:1: E402 module level import not at top of file
./tesdat/__init__.py:31:1: F401 'BLANK' imported but unused
./tesdat/__init__.py:31:1: F401 'ESCAPE' imported but unused
./tesdat/__init__.py:31:1: I100 Import statements are in the wrong order. from .api.special should be before from .formatters.csv
./tesdat/__init__.py:76:1: D205 1 blank line required between summary line and description
./tesdat/__init__.py:76:1: D400 First line should end with a period
./tesdat/__init__.py:88:9: F821 undefined name 'containers'
./tesdat/__init__.py:88:21: F821 undefined name 'models'
./tesdat/__init__.py:88:29: F821 undefined name 'patterns'
./tesdat/__init__.py:88:39: F821 undefined name 'formatters'
./tesdat/__init__.py:88:51: F821 undefined name 'exceptions'
./tesdat/__init__.py:88:63: F821 undefined name 'api'
./tesdat/compat.py:1:1: D100 Missing docstring in public module
./tesdat/compat.py:30:9: F401 'StringIO' imported but unused
./tesdat/exceptions.py:1:1: D100 Missing docstring in public module
./tesdat/exceptions.py:3:1: D101 Missing docstring in public class
./tesdat/exceptions.py:3:1: E302 expected 2 blank lines, found 1
./tesdat/exceptions.py:7:1: D101 Missing docstring in public class
./tesdat/exceptions.py:11:1: D101 Missing docstring in public class
./tesdat/exceptions.py:15:1: D101 Missing docstring in public class
./tesdat/api/__init__.py:1:1: D104 Missing docstring in public package
./tesdat/api/display.py:1:1: D100 Missing docstring in public module
./tesdat/api/display.py:5:1: D103 Missing docstring in public function
./tesdat/api/display.py:12:1: D103 Missing docstring in public function
./tesdat/api/display.py:22:1: W391 blank line at end of file
./tesdat/api/render.py:1:1: D100 Missing docstring in public module
./tesdat/api/render.py:7:1: I100 Import statements are in the wrong order. from .. should be before from ..patterns.base
./tesdat/api/render.py:9:1: E302 expected 2 blank lines, found 1
./tesdat/api/render.py:15:1: D205 1 blank line required between summary line and description
./tesdat/api/render.py:15:1: D400 First line should end with a period
./tesdat/api/render.py:37:80: E501 line too long (80 > 79 characters)
./tesdat/api/special.py:1:1: D100 Missing docstring in public module
./tesdat/api/special.py:4:1: D204 1 blank line required after class docstring
./tesdat/api/special.py:4:1: D400 First line should end with a period
./tesdat/api/special.py:6:1: D102 Missing docstring in public method
./tesdat/api/special.py:9:1: D105 Missing docstring in magic method
./tesdat/containers/__init__.py:1:1: D104 Missing docstring in public package
./tesdat/containers/base.py:1:1: D100 Missing docstring in public module
./tesdat/containers/base.py:9:1: D205 1 blank line required between summary line and description
./tesdat/containers/base.py:9:1: D400 First line should end with a period
./tesdat/containers/base.py:23:1: D200 One-line docstring should fit on one line with quotes
./tesdat/containers/base.py:23:1: D204 1 blank line required after class docstring
./tesdat/containers/base.py:26:1: D205 1 blank line required between summary line and description
./tesdat/containers/base.py:26:1: D400 First line should end with a period
./tesdat/containers/base.py:26:80: E501 line too long (90 > 79 characters)
./tesdat/containers/base.py:28:80: E501 line too long (87 > 79 characters)
./tesdat/containers/base.py:41:1: D102 Missing docstring in public method
./tesdat/containers/base.py:62:80: E501 line too long (81 > 79 characters)
./tesdat/containers/base.py:65:1: D102 Missing docstring in public method
./tesdat/containers/dict.py:1:1: D100 Missing docstring in public module
./tesdat/containers/dict.py:8:1: D101 Missing docstring in public class
./tesdat/containers/iter.py:1:1: D100 Missing docstring in public module
./tesdat/containers/iter.py:8:1: D200 One-line docstring should fit on one line with quotes
./tesdat/containers/iter.py:8:1: D204 1 blank line required after class docstring
./tesdat/containers/iter.py:8:1: D400 First line should end with a period
./tesdat/containers/iter.py:14:1: D105 Missing docstring in magic method
./tesdat/containers/iter.py:27:1: D102 Missing docstring in public method
./tesdat/containers/iter.py:30:1: D102 Missing docstring in public method
./tesdat/containers/iter.py:41:1: D105 Missing docstring in magic method
./tesdat/containers/list.py:1:1: D100 Missing docstring in public module
./tesdat/containers/list.py:8:1: D101 Missing docstring in public class
./tesdat/containers/list.py:12:80: E501 line too long (125 > 79 characters)
./tesdat/containers/list.py:28:1: D200 One-line docstring should fit on one line with quotes
./tesdat/containers/list.py:28:1: D400 First line should end with a period
./tesdat/containers/ordereddict.py:1:1: D100 Missing docstring in public module
./tesdat/containers/ordereddict.py:4:1: I100 Import statements are in the wrong order. from collections should be  before from .dict
./tesdat/containers/ordereddict.py:4:1: I201 Missing newline before sections or imports.
./tesdat/containers/ordereddict.py:7:1: D101 Missing docstring in public class
./tesdat/containers/ordereddict.py:8:1: D102 Missing docstring in public method
./tesdat/formatters/__init__.py:1:1: D104 Missing docstring in public package
./tesdat/formatters/base.py:1:1: D100 Missing docstring in public module
./tesdat/formatters/base.py:3:1: I100 Import statements are in the wrong order. import codecs should be before import os
./tesdat/formatters/base.py:8:1: D101 Missing docstring in public class
./tesdat/formatters/base.py:22:1: D102 Missing docstring in public method
./tesdat/formatters/csv.py:1:1: D100 Missing docstring in public module
./tesdat/formatters/csv.py:4:1: F401 'io' imported but unused
./tesdat/formatters/csv.py:5:1: F401 'os' imported but unused
./tesdat/formatters/csv.py:6:1: I100 Import statements are in the wrong order. import csv should be before import os
./tesdat/formatters/csv.py:9:1: F401 'OutputFileAlreadyExists' imported but unused
./tesdat/formatters/csv.py:9:1: I101 Imported names are in the wrong order. Should be CsvFormatterFieldsTypeInvalid, OutputFileAlreadyExists
./tesdat/formatters/csv.py:14:1: I100 Import statements are in the wrong order. from .. should be before from ..exceptions
./tesdat/formatters/csv.py:17:1: D101 Missing docstring in public class
./tesdat/formatters/csv.py:18:1: D102 Missing docstring in public method
./tesdat/formatters/csv.py:57:1: D102 Missing docstring in public method
./tesdat/formatters/json.py:1:1: D100 Missing docstring in public module
./tesdat/formatters/json.py:3:1: I201 Missing newline before sections or imports.
./tesdat/formatters/json.py:4:1: I101 Imported names are in the wrong order. Should be date, datetime, time
./tesdat/formatters/json.py:5:1: I201 Missing newline before sections or imports.
./tesdat/formatters/json.py:8:1: D200 One-line docstring should fit on one line with quotes
./tesdat/formatters/json.py:8:1: D204 1 blank line required after class docstring
./tesdat/formatters/json.py:18:1: D102 Missing docstring in public method
./tesdat/formatters/json.py:26:1: D101 Missing docstring in public class
./tesdat/formatters/json.py:27:1: D102 Missing docstring in public method
./tesdat/formatters/json.py:33:1: D102 Missing docstring in public method
./tesdat/formatters/pickle.py:1:1: D100 Missing docstring in public module
./tesdat/formatters/pickle.py:3:1: I201 Missing newline before sections or imports.
./tesdat/formatters/pickle.py:4:1: I201 Missing newline before sections or imports.
./tesdat/formatters/pickle.py:7:1: D101 Missing docstring in public class
./tesdat/formatters/pickle.py:8:1: D102 Missing docstring in public method
./tesdat/formatters/pickle.py:12:1: D102 Missing docstring in public method
./tesdat/formatters/string.py:1:1: D100 Missing docstring in public module
./tesdat/formatters/string.py:3:1: I201 Missing newline before sections or imports.
./tesdat/formatters/string.py:6:1: D101 Missing docstring in public class
./tesdat/formatters/string.py:7:1: D102 Missing docstring in public method
./tesdat/formatters/string.py:10:1: D102 Missing docstring in public method
./tesdat/models/__init__.py:1:1: D104 Missing docstring in public package
./tesdat/models/base.py:1:1: D100 Missing docstring in public module
./tesdat/models/base.py:4:1: D101 Missing docstring in public class
./tesdat/models/dict.py:1:1: D100 Missing docstring in public module
./tesdat/models/dict.py:6:1: D200 One-line docstring should fit on one line with quotes
./tesdat/models/dict.py:6:1: D204 1 blank line required after class docstring
./tesdat/models/dict.py:26:80: E501 line too long (83 > 79 characters)
./tesdat/models/dict.py:39:80: E501 line too long (88 > 79 characters)
./tesdat/models/dict.py:48:13: E265 block comment should start with '# '
./tesdat/models/list.py:1:1: D100 Missing docstring in public module
./tesdat/models/list.py:6:1: D200 One-line docstring should fit on one line with quotes
./tesdat/models/list.py:6:1: D204 1 blank line required after class docstring
./tesdat/models/list.py:40:1: D205 1 blank line required between summary line and description
./tesdat/models/list.py:44:80: E501 line too long (88 > 79 characters)
./tesdat/models/ordereddict.py:1:1: D100 Missing docstring in public module
./tesdat/models/ordereddict.py:3:1: I201 Missing newline before sections or imports.
./tesdat/models/ordereddict.py:6:1: D200 One-line docstring should fit on one line with quotes
./tesdat/models/ordereddict.py:6:1: D204 1 blank line required after class docstring
./tesdat/models/ordereddict.py:11:1: D102 Missing docstring in public method
./tesdat/patterns/__init__.py:1:1: D104 Missing docstring in public package
./tesdat/patterns/__init__.py:2:1: W391 blank line at end of file
./tesdat/patterns/base.py:1:1: D100 Missing docstring in public module
./tesdat/patterns/base.py:4:1: D101 Missing docstring in public class
./tesdat/patterns/choice.py:1:1: D100 Missing docstring in public module
./tesdat/patterns/choice.py:3:1: I201 Missing newline before sections or imports.
./tesdat/patterns/choice.py:7:1: D200 One-line docstring should fit on one line with quotes
./tesdat/patterns/choice.py:7:1: D204 1 blank line required after class docstring
./tesdat/patterns/choice.py:10:1: D205 1 blank line required between summary line and description
./tesdat/patterns/choice.py:10:1: D400 First line should end with a period
./tesdat/patterns/choice.py:24:1: D105 Missing docstring in magic method
./tesdat/patterns/choice.py:28:1: D400 First line should end with a period
./tesdat/patterns/cycle.py:1:1: D100 Missing docstring in public module
./tesdat/patterns/cycle.py:6:1: D200 One-line docstring should fit on one line with quotes
./tesdat/patterns/cycle.py:6:1: D204 1 blank line required after class docstring
./tesdat/patterns/cycle.py:9:1: D205 1 blank line required between summary line and description
./tesdat/patterns/cycle.py:9:1: D400 First line should end with a period
./tesdat/patterns/cycle.py:11:80: E501 line too long (95 > 79 characters)
./tesdat/patterns/cycle.py:14:80: E501 line too long (80 > 79 characters)
./tesdat/patterns/cycle.py:31:1: D105 Missing docstring in magic method
./tesdat/patterns/hashof.py:1:1: D100 Missing docstring in public module
./tesdat/patterns/hashof.py:10:1: D103 Missing docstring in public function
./tesdat/patterns/hashof.py:20:1: D200 One-line docstring should fit on one line with quotes
./tesdat/patterns/hashof.py:20:1: D204 1 blank line required after class docstring
./tesdat/patterns/hashof.py:23:1: D205 1 blank line required between summary line and description
./tesdat/patterns/hashof.py:23:80: E501 line too long (84 > 79 characters)
./tesdat/patterns/hashof.py:26:80: E501 line too long (92 > 79 characters)
./tesdat/patterns/hashof.py:28:80: E501 line too long (80 > 79 characters)
./tesdat/patterns/hashof.py:37:1: D105 Missing docstring in magic method
./tesdat/patterns/increment.py:1:1: D100 Missing docstring in public module
./tesdat/patterns/increment.py:7:1: D200 One-line docstring should fit on one line with quotes
./tesdat/patterns/increment.py:7:1: D204 1 blank line required after class docstring
./tesdat/patterns/increment.py:10:1: D205 1 blank line required between summary line and description
./tesdat/patterns/increment.py:14:80: E501 line too long (80 > 79 characters)
./tesdat/patterns/increment.py:21:1: D105 Missing docstring in magic method
./tesdat/patterns/pickout.py:1:1: D100 Missing docstring in public module
./tesdat/patterns/pickout.py:8:1: D200 One-line docstring should fit on one line with quotes
./tesdat/patterns/pickout.py:8:1: D204 1 blank line required after class docstring
./tesdat/patterns/pickout.py:11:1: D205 1 blank line required between summary line and description
./tesdat/patterns/pickout.py:11:1: D400 First line should end with a period
./tesdat/patterns/pickout.py:13:80: E501 line too long (93 > 79 characters)
./tesdat/patterns/pickout.py:15:80: E501 line too long (80 > 79 characters)
./tesdat/patterns/pickout.py:27:1: D105 Missing docstring in magic method
./tesdat/patterns/pickout.py:31:1: D400 First line should end with a period
./tesdat/patterns/sequence.py:1:1: D100 Missing docstring in public module
./tesdat/patterns/sequence.py:7:1: D200 One-line docstring should fit on one line with quotes
./tesdat/patterns/sequence.py:7:1: D204 1 blank line required after class docstring
./tesdat/patterns/sequence.py:10:1: D205 1 blank line required between summary line and description
./tesdat/patterns/sequence.py:10:1: D400 First line should end with a period
./tesdat/patterns/sequence.py:12:80: E501 line too long (95 > 79 characters)
./tesdat/patterns/sequence.py:15:80: E501 line too long (80 > 79 characters)
./tesdat/patterns/sequence.py:32:1: D105 Missing docstring in magic method
./tesdat/patterns/sequence.py:36:1: D400 First line should end with a period
./tesdat/tests/__init__.py:1:1: D104 Missing docstring in public package
./tesdat/tests/test_api.py:1:1: D100 Missing docstring in public module
./tesdat/tests/test_api.py:5:1: D101 Missing docstring in public class
./tesdat/tests/test_api.py:10:1: D102 Missing docstring in public method
./tesdat/tests/test_api.py:17:1: D102 Missing docstring in public method
./tesdat/tests/test_api.py:20:1: D102 Missing docstring in public method
./tesdat/tests/test_api.py:23:80: E501 line too long (84 > 79 characters)
./tesdat/tests/test_api.py:25:1: D102 Missing docstring in public method
./tesdat/tests/test_example.py:1:1: D100 Missing docstring in public module
./tesdat/tests/test_example.py:5:1: D101 Missing docstring in public class
./tesdat/tests/test_example.py:6:1: D102 Missing docstring in public method
./tesdat/tests/test_example.py:25:1: D102 Missing docstring in public method
./tesdat/tests/test_example.py:43:1: D102 Missing docstring in public method
./tesdat/tests/test_example.py:54:80: E501 line too long (81 > 79 characters)
./tesdat/tests/test_index.py:1:1: D100 Missing docstring in public module
./tesdat/tests/test_index.py:5:1: D101 Missing docstring in public class
./tesdat/tests/test_index.py:10:1: D102 Missing docstring in public method
./tesdat/tests/test_index.py:14:1: D102 Missing docstring in public method
./tesdat/tests/containers/__init__.py:1:1: D104 Missing docstring in public package
./tesdat/tests/containers/test_base.py:1:1: D100 Missing docstring in public module
./tesdat/tests/containers/test_base.py:5:1: D101 Missing docstring in public class
./tesdat/tests/containers/test_base.py:10:1: D102 Missing docstring in public method
./tesdat/tests/containers/test_base.py:15:1: D102 Missing docstring in public method
./tesdat/tests/containers/test_base.py:23:1: D101 Missing docstring in public class
./tesdat/tests/containers/test_base.py:24:1: D102 Missing docstring in public method
./tesdat/tests/containers/test_dict.py:1:1: D100 Missing docstring in public module
./tesdat/tests/containers/test_dict.py:5:1: D101 Missing docstring in public class
./tesdat/tests/containers/test_dict.py:10:1: D102 Missing docstring in public method
./tesdat/tests/containers/test_dict.py:25:1: D102 Missing docstring in public method
./tesdat/tests/containers/test_dict.py:40:1: D102 Missing docstring in public method
./tesdat/tests/containers/test_iter.py:1:1: D100 Missing docstring in public module
./tesdat/tests/containers/test_iter.py:5:1: D101 Missing docstring in public class
./tesdat/tests/containers/test_iter.py:10:1: D102 Missing docstring in public method
./tesdat/tests/containers/test_iter.py:14:1: D102 Missing docstring in public method
./tesdat/tests/containers/test_iter.py:25:1: D102 Missing docstring in public method
./tesdat/tests/containers/test_list.py:1:1: D100 Missing docstring in public module
./tesdat/tests/containers/test_list.py:5:1: D101 Missing docstring in public class
./tesdat/tests/containers/test_list.py:10:1: D102 Missing docstring in public method
./tesdat/tests/containers/test_list.py:25:1: D102 Missing docstring in public method
./tesdat/tests/containers/test_ordereddict.py:1:1: D100 Missing docstring in public module
./tesdat/tests/containers/test_ordereddict.py:5:1: D101 Missing docstring in public class
./tesdat/tests/containers/test_ordereddict.py:10:1: D102 Missing docstring in public method
./tesdat/tests/formatters/__init__.py:1:1: D104 Missing docstring in public package
./tesdat/tests/formatters/test_base.py:1:1: D100 Missing docstring in public module
./tesdat/tests/formatters/test_csv.py:1:1: D100 Missing docstring in public module
./tesdat/tests/formatters/test_csv.py:3:1: F401 'time' imported but unused
./tesdat/tests/formatters/test_csv.py:7:1: D101 Missing docstring in public class
./tesdat/tests/formatters/test_csv.py:10:1: D102 Missing docstring in public method
./tesdat/tests/formatters/test_csv.py:20:1: D102 Missing docstring in public method
./tesdat/tests/formatters/test_csv.py:33:1: D102 Missing docstring in public method
./tesdat/tests/formatters/test_csv.py:46:1: D102 Missing docstring in public method
./tesdat/tests/formatters/test_json.py:1:1: D100 Missing docstring in public module
./tesdat/tests/formatters/test_json.py:6:1: D101 Missing docstring in public class
./tesdat/tests/formatters/test_json.py:11:1: D102 Missing docstring in public method
./tesdat/tests/formatters/test_pickle.py:1:1: D100 Missing docstring in public module
./tesdat/tests/formatters/test_pickle.py:7:1: D101 Missing docstring in public class
./tesdat/tests/formatters/test_pickle.py:10:1: D102 Missing docstring in public method
./tesdat/tests/formatters/test_pickle.py:20:1: D102 Missing docstring in public method
./tesdat/tests/formatters/test_string.py:1:1: D100 Missing docstring in public module
./tesdat/tests/formatters/test_string.py:6:1: D101 Missing docstring in public class
./tesdat/tests/formatters/test_string.py:9:1: D102 Missing docstring in public method
./tesdat/tests/formatters/test_string.py:19:1: D102 Missing docstring in public method
./tesdat/tests/formatters/test_string.py:26:1: D102 Missing docstring in public method
./tesdat/tests/models/__init__.py:1:1: D104 Missing docstring in public package
./tesdat/tests/models/test_dict.py:1:1: D100 Missing docstring in public module
./tesdat/tests/models/test_dict.py:5:1: D103 Missing docstring in public function
./tesdat/tests/models/test_dict.py:9:1: D101 Missing docstring in public class
./tesdat/tests/models/test_dict.py:14:1: D102 Missing docstring in public method
./tesdat/tests/models/test_dict.py:21:1: D102 Missing docstring in public method
./tesdat/tests/models/test_dict.py:42:1: D102 Missing docstring in public method
./tesdat/tests/models/test_dict.py:56:1: D102 Missing docstring in public method
./tesdat/tests/models/test_dict.py:76:1: D102 Missing docstring in public method
./tesdat/tests/models/test_list.py:1:1: D100 Missing docstring in public module
./tesdat/tests/models/test_list.py:6:1: D103 Missing docstring in public function
./tesdat/tests/models/test_list.py:10:1: D101 Missing docstring in public class
./tesdat/tests/models/test_list.py:15:1: D102 Missing docstring in public method
./tesdat/tests/models/test_list.py:20:1: D102 Missing docstring in public method
./tesdat/tests/models/test_list.py:25:1: D102 Missing docstring in public method
./tesdat/tests/models/test_list.py:46:1: D102 Missing docstring in public method
./tesdat/tests/models/test_list.py:60:1: D102 Missing docstring in public method
./tesdat/tests/models/test_list.py:77:1: D102 Missing docstring in public method
./tesdat/tests/patterns/__init__.py:1:1: D104 Missing docstring in public package
./tesdat/tests/patterns/test_choice.py:1:1: D100 Missing docstring in public module
./tesdat/tests/patterns/test_choice.py:5:1: D101 Missing docstring in public class
./tesdat/tests/patterns/test_choice.py:10:1: D102 Missing docstring in public method
./tesdat/tests/patterns/test_choice.py:17:1: D102 Missing docstring in public method
./tesdat/tests/patterns/test_cycle.py:1:1: D100 Missing docstring in public module
./tesdat/tests/patterns/test_cycle.py:5:1: D101 Missing docstring in public class
./tesdat/tests/patterns/test_cycle.py:10:1: D102 Missing docstring in public method
./tesdat/tests/patterns/test_cycle.py:17:1: D102 Missing docstring in public method
./tesdat/tests/patterns/test_hashof.py:1:1: D100 Missing docstring in public module
./tesdat/tests/patterns/test_hashof.py:11:1: D101 Missing docstring in public class
./tesdat/tests/patterns/test_hashof.py:16:1: D102 Missing docstring in public method
./tesdat/tests/patterns/test_increment.py:1:1: D100 Missing docstring in public module
./tesdat/tests/patterns/test_increment.py:6:1: D101 Missing docstring in public class
./tesdat/tests/patterns/test_increment.py:11:1: D102 Missing docstring in public method
./tesdat/tests/patterns/test_increment.py:18:1: D102 Missing docstring in public method
./tesdat/tests/patterns/test_pickout.py:1:1: D100 Missing docstring in public module
./tesdat/tests/patterns/test_pickout.py:5:1: D101 Missing docstring in public class
./tesdat/tests/patterns/test_pickout.py:10:1: D102 Missing docstring in public method
./tesdat/tests/patterns/test_pickout.py:19:1: D102 Missing docstring in public method
./tesdat/tests/patterns/test_sequence.py:1:1: D100 Missing docstring in public module
./tesdat/tests/patterns/test_sequence.py:5:1: D101 Missing docstring in public class
./tesdat/tests/patterns/test_sequence.py:10:1: D102 Missing docstring in public method
./tesdat/tests/patterns/test_sequence.py:19:1: D102 Missing docstring in public method
./tesdat/tests/utils/__init__.py:1:1: D104 Missing docstring in public package
./tesdat/tests/utils/test_datetime.py:1:1: D100 Missing docstring in public module
./tesdat/tests/utils/test_datetime.py:3:1: I100 Import statements are in the wrong order. from datetime should be before from unittest
./tesdat/tests/utils/test_datetime.py:9:1: D101 Missing docstring in public class
./tesdat/tests/utils/test_datetime.py:14:1: D102 Missing docstring in public method
./tesdat/tests/utils/test_datetime.py:20:1: D102 Missing docstring in public method
./tesdat/tests/utils/test_datetime.py:30:1: D102 Missing docstring in public method
./tesdat/tests/utils/test_datetime.py:43:1: D102 Missing docstring in public method
./tesdat/tests/utils/test_datetime.py:59:1: D101 Missing docstring in public class
./tesdat/tests/utils/test_datetime.py:64:1: D102 Missing docstring in public method
./tesdat/tests/utils/test_datetime.py:73:80: E501 line too long (85 > 79 characters)
./tesdat/tests/utils/test_datetime.py:76:1: D101 Missing docstring in public class
./tesdat/tests/utils/test_datetime.py:81:1: D102 Missing docstring in public method
./tesdat/tests/utils/test_datetime.py:85:1: D102 Missing docstring in public method
./tesdat/utils/__init__.py:1:1: D104 Missing docstring in public package
./tesdat/utils/datetime.py:1:1: D100 Missing docstring in public module
./tesdat/utils/datetime.py:6:1: I101 Imported names are in the wrong order. Should be date, datetime, time, timedelta
./tesdat/utils/datetime.py:20:1: E402 module level import not at top of file
./tesdat/utils/datetime.py:63:1: D400 First line should end with a period
./tesdat/utils/datetime.py:87:1: D400 First line should end with a period
ERROR: InvocationError: '/home/vagrant/tesdat/.tox/flake8/bin/flake8 .'
____________________________________________________ summary _____________________________________________________
ERROR:   py27: commands failed
ERROR:   py33: commands failed
ERROR:   py34: commands failed
ERROR:   py35: commands failed
ERROR:   flake8: commands failed

ってうわー、すごいエラー(警告)が。

実はエラーに関しては前から気付いていて、辞書の値が特定の順番になることを前提にテストを書いていたため、エラーになっています。
また脱線しますが、これは実行時に決まるPYTHONHASHSEEDという環境変数に左右されます。
この場合は「PYTHONHASHSEED=’115669612’」であればテストは通るのです。

(venv3.5) [vagrant@localhost tesdat]$ PYTHONHASHSEED='115669612' tox
~~~ 中略 ~~~
___________________________________________________________ summary ___________________________________________________________
py27: commands succeeded
py33: commands succeeded
py34: commands succeeded
py35: commands succeeded
ERROR:   flake8: commands failed

とりあえず、テストを直します。興味ないかもしれませんが、こんな感じ。

diff --git a/tesdat/tests/formatters/test_json.py b/tesdat/tests/formatters/test_json.py
index 19f2542..796672b 100644
--- a/tesdat/tests/formatters/test_json.py
+++ b/tesdat/tests/formatters/test_json.py
@@ -1,4 +1,5 @@
 # coding: utf-8
+import json
 from datetime import time
 from unittest import TestCase
 
@@ -11,8 +12,8 @@ class TestJsonFormatter(TestCase):
     def test_stringify(self):
         sf = self._getClass({'a': time(23, 59), 'b': ('2',)}, indent=None)
         self.assertEqual(
-            sf.stringify(),
-            '{"a": "23:59:00", "b": ["2"]}'
+            json.loads(sf.stringify()),
+            {"a": "23:59:00", "b": ["2"]}
         )
 
         sf = self._getClass({list}, indent=None)
diff --git a/tesdat/tests/formatters/test_string.py b/tesdat/tests/formatters/test_string.py
index 372b6c7..94c025c 100644
--- a/tesdat/tests/formatters/test_string.py
+++ b/tesdat/tests/formatters/test_string.py
@@ -19,8 +19,8 @@ class TestStringFormatter(TestCase):
     def test_string(self):
         sf = self._getClass({'a': 1, 'b': ('2',)})
         self.assertEqual(
-            sf.stringify(),
-            "{'a': 1, 'b': ('2',)}"
+            eval(sf.stringify()),
+            {'a': 1, 'b': ('2',)}
         )

先ほどは失敗したPYTHONHASHSEEDで通ることを確認しました。

(venv3.5) [vagrant@localhost tesdat]$ $ PYTHONHASHSEED='1127406230' tox
~~~ 中略 ~~~
___________________________________________________________ summary ___________________________________________________________
py27: commands succeeded
py33: commands succeeded
py34: commands succeeded
py35: commands succeeded
ERROR:   flake8: commands failed

ちなみにPYTHONHASHSEEDに限らず環境変数を固定したい場合、tox.iniの[testenv]セクションに「setenv」を指定します。

[testenv]
setenv =
  PYTHONHASHSEED = 1127406230

flake8

はい、ではいい加減flake8に手を付けます。
この中には無視したいものもあるはずです。吟味したうえで要らないものは除外設定しましょう。
それぞれ何を言われているのでしょうか。さすがに全部に表にするのはしんどいので簡単にまとめました。

CODE 対応 説明(英語) 要約
I100 無視 Import statements are in the wrong order. importするモジュールの順序が不正。ただどうすればいいのかはよくわからない。
I101 無視 Imported names are in the wrong order. Should be xxx, yyy, zzz import順がアルファベット順になっているべき。
I201 直す Missing newline before sections or imports. importする対象のモジュール種類によって空行を入れるべき。
E226 直す missing whitespace around arithmetic operator 演算子の前後に空行が必要。
E265 直す block comment should start with ‘# ‘ コメント記号「#」と文字列との間に空白が1つ必要。
E302 直す expected 2 blank lines, found x 関数やクラスの定義は2行空けるべき。
E402 直す module level import not at top of file これは__init__.pyで発生してる模様なので、このファイルをflake8の対象から除外。
E501 直す line too long (xx > 79 characters) よく見る1行当たりの文字数オーバー。個人で開発してるのにこんなこと気にしたくないです。とはいえ長すぎるのも考え物なので倍の160文字にしましょう。
F401 直す imported but unused importされてるけど使われてない。本来であれば気にするんですが、このライブラリでは定義されていてユーザに使われる目的の変数もあるのです。なので、__init__.pyをflake8の対象から除外することで対応する。
F821 直す undefined name ‘xxx’ 変数が定義されていない。これは__init__.pyで発生してる模様なので、flake8の対象から除外することで対応する。
D100 無視 Missing docstring in public module 公開モジュールにdocstringで説明がない。ちなみにD開始はdocstring関連です。ワンピースは関係ない。
D101 無視 Missing docstring in public class 公開クラスにdocstringがない。(以下略
D103 無視 Missing docstring in public function 公開メソッドに(以下略
D104 無視 Missing docstring in public package 公開パッケージに(以下略
D105 無視 Missing docstring in magic method マジックメソッドに(以下略
D200 直す One-line docstring should fit on one line with quotes 一行のdocstringは改行なしで表示しよう。
D204 直す 1 blank line required after class docstring クラスレベルのdocstingの後には1行空行が必要。
D205 直す 1 blank line required between summary line and description docstringの前後に不要な改行がある。見やすいように改行を入れた改行がよくなかったらしい。
D210 直す No whitespaces allowed surrounding docstring text docstringの前後には空白がいらない。
D400 無視 First line should end with a period docstringの先頭行の末尾は「.(ドット)」で終わってないといけない。
W391 直す blank line at end of file ファイルの末尾に余計な空行がある。はい、直します。

こんな感じで対応します。反映させたtox.iniが以下です。

[flake8]
ignore = D100, D101, D102, D103, D104, D105, D400, I100, I101
exclude = tests/*, tesdat/__init__.py, tesdat/compat.py
max-line-length = 160

もちろんコードも直しました。

.toxディレクトリ

一度toxを実行すると、「.tox/」配下にenvlist毎の設定を作成し、次回以降は再利用されます。

(venv3.5) [vagrant@localhost tesdat]$ ls .tox/
dist  flake8  log  py27  py33  py34  py35

一旦初期化したい場合は「-r」「–recreate」オプションを指定しましょう。
たまに前の実行結果に引きずられてテストが落ちたりすることがあるので、ハマったら一度試してみましょう。

カバレッジとる

ついでなのでカバレッジを取りましょう。

tox.iniを以下のように編集します。

[testenv]
commands =
  py.test --cov
deps =
  pytest-cov

「pytest-cov」をインストールし、–covオプションを指定します。これにより、実行時にカバレッジが取得されます。

「$ coverage html」を実行すると(htmlcov/に)HTMLが作成されるのでお好みで実行してください。

で、これだと検査したくないファイルまで対象となってしまうため、coveragercによって除外します。

[run]
omit = .tox/*, setup.py, tesdat/tests/*

あとリポジトリに入らないように.gitignoreに以下を追加します。

.coverage
coverage.xml
htmlcov/*

実行してみましょう。

(venv3.5) [vagrant@localhost tesdat]$ tox -r
GLOB sdist-make: /home/vagrant/tesdat/setup.py
py27 recreate: /home/vagrant/tesdat/.tox/py27
py27 installdeps: pytest-cov
py27 inst: /home/vagrant/tesdat/.tox/dist/tesdat-2.1.1.zip
py27 installed: coverage==4.0.3,py==1.4.31,pytest==2.9.1,pytest-cov==2.2.1,tesdat==2.1.1
py27 runtests: PYTHONHASHSEED='2232003898'
py27 runtests: commands[0] | py.test --cov
================================================= test session starts =================================================
platform linux2 -- Python 2.7.10, pytest-2.9.1, py-1.4.31, pluggy-0.3.1
rootdir: /home/vagrant/tesdat, inifile:
plugins: cov-2.2.1
collected 58 items

tests-require.txt s
tesdat/tests/test_api.py ....
tesdat/tests/test_example.py ...
tesdat/tests/test_index.py ..
tesdat/tests/containers/test_base.py ...
tesdat/tests/containers/test_dict.py ...
tesdat/tests/containers/test_iter.py ...
tesdat/tests/containers/test_list.py ..
tesdat/tests/containers/test_ordereddict.py .
tesdat/tests/formatters/test_csv.py ...
tesdat/tests/formatters/test_json.py .
tesdat/tests/formatters/test_pickle.py .
tesdat/tests/formatters/test_string.py ..
tesdat/tests/models/test_dict.py .....
tesdat/tests/models/test_list.py ......
tesdat/tests/patterns/test_choice.py ..
tesdat/tests/patterns/test_cycle.py ..
tesdat/tests/patterns/test_hashof.py .
tesdat/tests/patterns/test_increment.py ..
tesdat/tests/patterns/test_pickout.py ..
tesdat/tests/patterns/test_sequence.py ..
tesdat/tests/utils/test_datetime.py .......
---------------------------------- coverage: platform linux2, python 2.7.10-final-0 -----------------------------------
Name                               Stmts   Miss  Cover
------------------------------------------------------
tesdat/__init__.py                    45      8    82%
tesdat/api/__init__.py                 0      0   100%
tesdat/api/display.py                 15      1    93%
tesdat/api/render.py                  23      4    83%
tesdat/api/special.py                  7      1    86%
tesdat/compat.py                      37     18    51%
tesdat/containers/__init__.py          0      0   100%
tesdat/containers/base.py             37      3    92%
tesdat/containers/dict.py             17      0   100%
tesdat/containers/iter.py             31      0   100%
tesdat/containers/list.py             19      1    95%
tesdat/containers/ordereddict.py       6      0   100%
tesdat/exceptions.py                   8      0   100%
tesdat/formatters/__init__.py          0      0   100%
tesdat/formatters/base.py             17      1    94%
tesdat/formatters/csv.py              42      5    88%
tesdat/formatters/json.py             20      0   100%
tesdat/formatters/pickle.py            9      0   100%
tesdat/formatters/string.py            7      0   100%
tesdat/models/__init__.py              0      0   100%
tesdat/models/base.py                  2      0   100%
tesdat/models/dict.py                 27      1    96%
tesdat/models/list.py                 34      1    97%
tesdat/patterns/__init__.py            0      0   100%
tesdat/patterns/base.py                3      0   100%
tesdat/patterns/choice.py             18      1    94%
tesdat/patterns/cycle.py              26      1    96%
tesdat/patterns/hashof.py             20      3    85%
tesdat/patterns/increment.py          16      2    88%
tesdat/patterns/pickout.py            25      2    92%
tesdat/patterns/sequence.py           30      1    97%
tesdat/utils/__init__.py               0      0   100%
tesdat/utils/datetime.py              48      6    88%
------------------------------------------------------
TOTAL                                589     60    90%

======================================== 57 passed, 1 skipped in 0.82 seconds =========================================
py27 runtests: commands[1] | coverage html
py33 recreate: /home/vagrant/tesdat/.tox/py33
py33 installdeps: pytest-cov
py33 inst: /home/vagrant/tesdat/.tox/dist/tesdat-2.1.1.zip
py33 installed: coverage==4.0.3,py==1.4.31,pytest==2.9.1,pytest-cov==2.2.1,tesdat==2.1.1
py33 runte
sts: PYTHONHASHSEED='2232003898'
py33 runtests: commands[0] | py.test --cov
================================================= test session starts =================================================
platform linux -- Python 3.3.6, pytest-2.9.1, py-1.4.31, pluggy-0.3.1
rootdir: /home/vagrant/tesdat, inifile:
plugins: cov-2.2.1
collected 58 items

tests-require.txt s
tesdat/tests/test_api.py ....
tesdat/tests/test_example.py ...
tesdat/tests/test_index.py ..
tesdat/tests/containers/test_base.py ...
tesdat/tests/containers/test_dict.py ...
tesdat/tests/containers/test_iter.py ...
tesdat/tests/containers/test_list.py ..
tesdat/tests/containers/test_ordereddict.py .
tesdat/tests/formatters/test_csv.py ...
tesdat/tests/formatters/test_json.py .
tesdat/tests/formatters/test_pickle.py .
tesdat/tests/formatters/test_string.py ..
tesdat/tests/models/test_dict.py .....
tesdat/tests/models/test_list.py ......
tesdat/tests/patterns/test_choice.py ..
tesdat/tests/patterns/test_cycle.py ..
tesdat/tests/patterns/test_hashof.py .
tesdat/tests/patterns/test_increment.py ..
tesdat/tests/patterns/test_pickout.py ..
tesdat/tests/patterns/test_sequence.py ..
tesdat/tests/utils/test_datetime.py .......
----------------------------------- coverage: platform linux, python 3.3.6-final-0 ------------------------------------
Name                               Stmts   Miss  Cover
------------------------------------------------------
tesdat/__init__.py                    45      8    82%
tesdat/api/__init__.py                 0      0   100%
tesdat/api/display.py                 15      1    93%
tesdat/api/render.py                  23      4    83%
tesdat/api/special.py                  7      1    86%
tesdat/compat.py                      37      4    89%
tesdat/containers/__init__.py          0      0   100%
tesdat/containers/base.py             37      3    92%
tesdat/containers/dict.py             17      0   100%
tesdat/containers/iter.py             31      0   100%
tesdat/containers/list.py             19      1    95%
tesdat/containers/ordereddict.py       6      0   100%
tesdat/exceptions.py                   8      0   100%
tesdat/formatters/__init__.py          0      0   100%
tesdat/formatters/base.py             17      1    94%
tesdat/formatters/csv.py              42      5    88%
tesdat/formatters/json.py             20      0   100%
tesdat/formatters/pickle.py            9      0   100%
tesdat/formatters/string.py            7      0   100%
tesdat/models/__init__.py              0      0   100%
tesdat/models/base.py                  2      0   100%
tesdat/models/dict.py                 27      1    96%
tesdat/models/list.py                 34      1    97%
tesdat/patterns/__init__.py            0      0   100%
tesdat/patterns/base.py                3      0   100%
tesdat/patterns/choice.py             18      1    94%
tesdat/patterns/cycle.py              26      1    96%
tesdat/patterns/hashof.py             20      3    85%
tesdat/patterns/increment.py          16      2    88%
tesdat/patterns/pickout.py            25      2    92%
tesdat/patterns/sequence.py           30      1    97%
tesdat/utils/__init__.py               0      0   100%
tesdat/utils/datetime.py              48      6    88%
------------------------------------------------------
TOTAL                                589     46    92%

======================================== 57 passed, 1 skipped in 1.09 seconds =========================================
py33 runtests: commands[1] | coverage html
py34 recreate: /home/vagrant/tesdat/.tox/py34
py34 installdeps: pytest-cov
py34 inst: /home/vagrant/tesdat/.tox/dist/tesdat-2.1.1.zip
py34 installed: coverage==4.0.3,py==1.4.31,pytest==2.9.1,pytest-cov==2.2.1,tesdat==2.1.1
py34 runtests: PYTHONHASHSEED='2232003898'
py34 runtests: commands[0] | py.test --cov
================================================= test session starts =================================================
platform linux -- Python 3.4.4, pytest-2.9.1, py-1.4.31, pluggy-0.3.1
rootdir: /home/vagrant/tesdat, inifile:
plugins: cov-2.2.1
collected 58 items

tests-require.txt s
tesdat/tests/test_api.py ....
tesdat/tests/test_example.py ...
tesdat/tests/test_index.py ..
tesdat/tests/containers/test_base.py ...
tesdat/tests/containers/test_dict.py ...
tesdat/tests/containers/test_iter.py ...
tesdat/tests/containers/test_list.py ..
tesdat/tests/containers/test_ordereddict.py .
tesdat/tests/formatters/test_csv.py ...
tesdat/tests/formatters/test_json.py .
tesdat/tests/formatters/test_pickle.py .
tesdat/tests/formatters/test_string.py ..
tesdat/tests/models/test_dict.py .....
tesdat/tests/models/test_list.py ......
tesdat/tests/patterns/test_choice.py ..
tesdat/tests/patterns/test_cycle.py ..
tesdat/tests/patterns/test_hashof.py .
tesdat/tests/patterns/test_increment.py ..
tesdat/tests/patterns/test_pickout.py ..
tesdat/tests/patterns/test_sequence.py ..
tesdat/tests/utils/test_datetime.py .......
----------------------------------- coverage: platform linux, python 3.4.4-final-0 ------------------------------------
Name                               Stmts   Miss  Cover
------------------------------------------------------
tesdat/__init__.py                    45      8    82%
tesdat/api/__init__.py                 0      0   100%
tesdat/api/display.py                 15      1    93%
tesdat/api/render.py                  23      4    83%
tesdat/api/special.py                  7      1    86%
tesdat/compat.py                      37      4    89%
tesdat/containers/__init__.py          0      0   100%
tesdat/containers/base.py             37      3    92%
tesdat/containers/dict.py             17      0   100%
tesdat/containers/iter.py             31      0   100%
tesdat/containers/list.py             19      1    95%
tesdat/containers/ordereddict.py       6      0   100%
tesdat/exceptions.py                   8      0   100%
tesdat/formatters/__init__.py          0      0   100%
tesdat/formatters/base.py             17      1    94%
tesdat/formatters/csv.py              42      5    88%
tesdat/formatters/json.py             20      0   100%
tesdat/formatters/pickle.py            9      0   100%
tesdat/formatters/string.py            7      0   100%
tesdat/models/__init__.py              0      0   100%
tesdat/models/base.py                  2      0   100%
tesdat/models/dict.py                 27      1    96%
tesdat/models/list.py                 34      1    97%
tesdat/patterns/__init__.py            0      0   100%
tesdat/patterns/base.py                3      0   100%
tesdat/patterns/choice.py             18      1    94%
tesdat/patterns/cycle.py              26      1    96%
tesdat/patterns/hashof.py             20      3    85%
tesdat/patterns/increment.py          16      2    88%
tesdat/patterns/pickout.py            25      2    92%
tesdat/patterns/sequence.py           30      1    97%
tesdat/utils/__init__.py               0      0   100%
tesdat/utils/datetime.py              48      6    88%
------------------------------------------------------
TOTAL                                589     46    92%

======================================== 57 passed, 1 skipped in 1.14 seconds =========================================
py34 runtests: commands[1] | coverage html
py35 recreate: /home/vagrant/tesdat/.tox/py35
py35 installdeps: pytest-cov
py35 inst: /home/vagrant/tesdat/.tox/dist/tesdat-2.1.1.zip
py35 installed: coverage==4.0.3,py==1.4.31,pytest==2.9.1,pytest-cov==2.2.1,tesdat==2.1.1
py35 runtests: PYTHONHASHSEED='2232003898'
py35 runtests: commands[0] | py.test --cov
================================================= test session starts =================================================
platform linux -- Python 3.5.1, pytest-2.9.1, py-1.4.31, pluggy-0.3.1
rootdir: /home/vagrant/tesdat, inifile:
plugins: cov-2.2.1
collected 58 items

tests-require.txt s
tesdat/tests/test_api.py ....
tesdat/tests/test_example.py ...
tesdat/tests/test_index.py ..
tesdat/tests/containers/test_base.py ...
tesdat/tests/containers/test_dict.py ...
tesdat/tests/containers/test_iter.py ...
tesdat/tests/containers/test_list.py ..
tesdat/tests/containers/test_ordereddict.py .
tesdat/tests/formatters/test_csv.py ...
tesdat/tests/formatters/test_json.py .
tesdat/tests/formatters/test_pickle.py .
tesdat/tests/formatters/test_string.py ..
tesdat/tests/models/test_dict.py .....
tesdat/tests/models/test_list.py ......
tesdat/tests/patterns/test_choice.py ..
tesdat/tests/patterns/test_cycle.py ..
tesdat/tests/patterns/test_hashof.py .
tesdat/tests/patterns/test_increment.py ..
tesdat/tests/patterns/test_pickout.py ..
tesdat/tests/patterns/test_sequence.py ..
tesdat/tests/utils/test_datetime.py .......
----------------------------------- coverage: platform linux, python 3.5.1-final-0 ------------------------------------
Name                               Stmts   Miss  Cover
------------------------------------------------------
tesdat/__init__.py                    45      8    82%
tesdat/api/__init__.py                 0      0   100%
tesdat/api/display.py                 15      1    93%
tesdat/api/render.py                  23      4    83%
tesdat/api/special.py                  7      1    86%
tesdat/compat.py                      37      4    89%
tesdat/containers/__init__.py          0      0   100%
tesdat/containers/base.py             37      3    92%
tesdat/containers/dict.py             17      0   100%
tesdat/containers/iter.py             31      0   100%
tesdat/containers/list.py             19      1    95%
tesdat/containers/ordereddict.py       6      0   100%
tesdat/exceptions.py                   8      0   100%
tesdat/formatters/__init__.py          0      0   100%
tesdat/formatters/base.py             17      1    94%
tesdat/formatters/csv.py              42      5    88%
tesdat/formatters/json.py             20      0   100%
tesdat/formatters/pickle.py            9      0   100%
tesdat/formatters/string.py            7      0   100%
tesdat/models/__init__.py              0      0   100%
tesdat/models/base.py                  2      0   100%
tesdat/models/dict.py                 27      1    96%
tesdat/models/list.py                 34      1    97%
tesdat/patterns/__init__.py            0      0   100%
tesdat/patterns/base.py                3      0   100%
tesdat/patterns/choice.py             18      1    94%
tesdat/patterns/cycle.py              26      1    96%
tesdat/patterns/hashof.py             20      3    85%
tesdat/patterns/increment.py          16      2    88%
tesdat/patterns/pickout.py            25      2    92%
tesdat/patterns/sequence.py           30      1    97%
tesdat/utils/__init__.py               0      0   100%
tesdat/utils/datetime.py              48      6    88%
------------------------------------------------------
TOTAL                                589     46    92%

======================================== 57 passed, 1 skipped in 1.08 seconds =========================================
py35 runtests: commands[1] | coverage html
flake8 recreate: /home/vagrant/tesdat/.tox/flake8
flake8 installdeps: flake8, flake8-blind-except, flake8-docstrings, flake8-import-order, mccabe, radon
flake8 inst: /home/vagrant/tesdat/.tox/dist/tesdat-2.1.1.zip
flake8 installed: colorama==0.3.7,flake8==2.5.4,flake8-blind-except==0.1.0,flake8-docstrings==0.2.6,flake8-import-order==0.7,mando==0.3.3,mccabe==0.4.0,pep257==0.7.0,pep8==1.7.0,pyflakes==1.0.0,radon==1.3.2,tesdat==2.1.1
flake8 runtests: PYTHONHASHSEED='2232003898'
flake8 runtests: commands[0] | flake8 .
_______________________________________________________ summary _______________________________________________________
  py27: commands succeeded
  py33: commands succeeded
  py34: commands succeeded
  py35: commands succeeded
  flake8: commands succeeded
  congratulations :)

htmlcov/index.htmlにアクセスすると以下のように表示されます。
coverage

埋め込める変数

上では説明しませんでしたが、tox.ini中には「{spam}」のように変数を埋め込むことができます。
詳細はこちらで解説されていますが、重要そうな2つの変数については軽く動作を見ておきましょう。

posargs

posargsはtoxから渡された引数を参照します。
例えばtox.iniのコマンドを以下のように書き換え

[testenv]
commands =
  py.test --cov . {posargs}

「tox -e py35 tesdat/tests/test_api.py」のように実行すると、テストコマンドとしては「py.test –cov . tesdat/tests/test_api.py」のように展開されます。
pytestコマンドのテスト対象についてはこちらで解説しています。

「–」から始まる引数はtoxのオプションとして認識され、そんなのないよってことでエラーになっちゃうので注意してください。

(venv3.5) [vagrant@localhost tesdat]$ tox --aa
usage: tox [--version] [-h] [--help-ini] [-v] [--showconfig] [-l]
           [-c CONFIGFILE] [-e envlist] [--notest] [--sdistonly]
           [--installpkg PATH] [--develop] [-i URL] [--pre] [-r]
           [--result-json PATH] [--hashseed SEED] [--force-dep REQ]
           [--sitepackages] [--skip-missing-interpreters]
           [args [args ...]]
tox: error: unrecognized arguments: --aa

toxinidir

この変数はtox.iniがおいてあるディレクトリのパスを示します。
どこかに移動してからテストを実行したいとかであれば以下のように書けますね。

[testenv]
changedir = {toxinidir}/apps/
commands =
  py.test --cov .


toxについて記述がある数少ない書籍です。

次はtox x vagrantでハマったことについて書きます。