2016-06-20

[Python] え、まだtox使ってないんですか？

今回はtoxです。さんざん記事書くって言ってあったので書きます (本当は4月の予定だったけどね

この記事は神々(aodag, shimzukawa)の知識をほとんどそのまま拝借したものです。ほぼ乗っ取られてます。

準備

今回テストの対象とするのは私が公開してる tesdat というライブラリを使います。

もちろん試したいコードがある方はそちらでよいです。

複数のバージョンのpython がインストールされている VM内で git clone します。

[vagrant@localhost ~]$ cd projects/ # projects配下はホスト間と共有されてる
[vagrant@localhost projects]$ git clone https://github.com/righ/tesdat
[vagrant@localhost projects]$ cd tesdat/

準備はこれで終了。

導入

ではインストール。このときvenvを有効にしてもしなくてもOKです。私はやりますが。

$ pip install tox

残念ながらまだ tox は使えません。

(venv3.5) [vagrant@localhost tesdat]$ tox
ERROR: toxini file 'tox.ini' not found

tox を使うには tox.ini という設定ファイルが必要です。ひとまずこんな風にしましょう。

[tox]
envlist = py27, py33, py34, py35, flake8

[testenv]
commands = py.test
deps =
  pytest

[testenv:flake8]
deps =
    flake8
    flake8-blind-except
    flake8-docstrings
    flake8-import-order

commands = flake8 .

[flake8]
exclude = tests/*

簡単に説明すると

テストランナーは py.test を使う
python2.7, python3.3, python3.4, python3.5 でテストを実施する。
- python3.2(というかpip3.2?)がtoxでうまく動作しなかったので3.3からにしました。しょうがないね。
tests/ フォルダを除いてflake8を実施する。
flake8 での構文を確認するためにライブラリをインストールする

って感じですね。

複数のコマンドを実行したい場合は deps と同じように commands を複数行にすればOKです。

tox.ini で特に重要なのは tox と testenv セクションです。

「tox」セクションの「envlist」に記述したコマンドが実際に何をするかやオプションを testenv:~ に記述します。

info

デフォルトenvは testenv セクションに属します。
簡単に言うと py*(python*), jython, pypy です。
詳しくはこちらを参照。

特定の環境だけでテストを実行したい場合は -e オプションを使います。

例えば上記設定を用いてflake8だけを実行したい場合は tox-e flake8 のようにします。

deps にはテストに必要なライブラリを requirements.txt 形式で指定できます。

すでにプロジェクトで requirements.txt がある場合 -rrequirements.txt のように指定できます。テスト用の requirements.txt を作っておくのがよいでしょう。

とりあえず実行してみましょう。

(venv3.5) [vagrant@localhost tesdat]$ tox
GLOB sdist-make: /home/vagrant/tesdat/setup.py
py27 inst-nodeps: /home/vagrant/tesdat/.tox/dist/tesdat-2.1.1.zip
py27 installed: tesdat==2.1.1
py27 runtests: PYTHONHASHSEED='1127406230'
py27 runtests: commands[0] | py.test
WARNING:test command found but not installed in testenv
  cmd: /home/vagrant/venv3.5/bin/py.test
  env: /home/vagrant/tesdat/.tox/py27
Maybe you forgot to specify a dependency? See also the whitelist_externals envconfig setting.
WARNING: Attempting to work in a virtualenv. If you encounter problems, please install IPython inside the virtualenv.
WARNING: IPython History requires SQLite, your history will not be saved
============================================== test session starts ===============================================
platform linux -- Python 3.5.1, pytest-2.9.1, py-1.4.31, pluggy-0.3.1
rootdir: /home/vagrant/tesdat, inifile:
plugins: ipdb-0.1.dev2, django-2.9.1
collected 57 items

tesdat/tests/test_api.py ....
tesdat/tests/test_example.py ...
tesdat/tests/test_index.py ..
tesdat/tests/containers/test_base.py ...
tesdat/tests/containers/test_dict.py ...
tesdat/tests/containers/test_iter.py ...
tesdat/tests/containers/test_list.py ..
tesdat/tests/containers/test_ordereddict.py .
tesdat/tests/formatters/test_csv.py ...
tesdat/tests/formatters/test_json.py F
tesdat/tests/formatters/test_pickle.py .
tesdat/tests/formatters/test_string.py F.
tesdat/tests/models/test_dict.py .....
tesdat/tests/models/test_list.py ......
tesdat/tests/patterns/test_choice.py ..
tesdat/tests/patterns/test_cycle.py ..
tesdat/tests/patterns/test_hashof.py .
tesdat/tests/patterns/test_increment.py ..
tesdat/tests/patterns/test_pickout.py ..
tesdat/tests/patterns/test_sequence.py ..
tesdat/tests/utils/test_datetime.py .......

==================================================== FAILURES ====================================================
________________________________________ TestJsonFormatter.test_stringify ________________________________________

self =

    def test_stringify(self):
        sf = self._getClass({'a': time(23, 59), 'b': ('2',)}, indent=None)
        self.assertEqual(
            sf.stringify(),
>           '{"a": "23:59:00", "b": ["2"]}'
        )
E       AssertionError: '{"b": ["2"], "a": "23:59:00"}' != '{"a": "23:59:00", "b": ["2"]}'
E       - {"b": ["2"], "a": "23:59:00"}
E       + {"a": "23:59:00", "b": ["2"]}

tesdat/tests/formatters/test_json.py:15: AssertionError
________________________________________ TestStringFormatter.test_string _________________________________________

self =

    def test_string(self):
        sf = self._getClass({'a': 1, 'b': ('2',)})
        self.assertEqual(
            sf.stringify(),
>           "{'a': 1, 'b': ('2',)}"
        )
E       AssertionError: "{'b': ('2',), 'a': 1}" != "{'a': 1, 'b': ('2',)}"
E       - {'b': ('2',), 'a': 1}
E       + {'a': 1, 'b': ('2',)}

tesdat/tests/formatters/test_string.py:23: AssertionError
====================================== 2 failed, 55 passed in 0.95 seconds =======================================
ERROR: InvocationError: '/home/vagrant/venv3.5/bin/py.test'
py33 inst-nodeps: /home/vagrant/tesdat/.tox/dist/tesdat-2.1.1.zip
py33 installed: tesdat==2.1.1
y33 runtests: PYTHONHASHSEED='1127406230'
py33 runtests: commands[0] | py.test
WARNING:test command found but not installed in testenv
  cmd: /home/vagrant/venv3.5/bin/py.test
  env: /home/vagrant/tesdat/.tox/py33
Maybe you forgot to specify a dependency? See also the whitelist_externals envconfig setting.
WARNING: Attempting to work in a virtualenv. If you encounter problems, please install IPython inside the virtualenv.
WARNING: IPython History requires SQLite, your history will not be saved
============================================== test session starts ===============================================
platform linux -- Python 3.5.1, pytest-2.9.1, py-1.4.31, pluggy-0.3.1
rootdir: /home/vagrant/tesdat, inifile:
plugins: ipdb-0.1.dev2, django-2.9.1
collected 57 items

tesdat/tests/test_api.py ....
tesdat/tests/test_example.py ...
tesdat/tests/test_index.py ..
tesdat/tests/containers/test_base.py ...
tesdat/tests/containers/test_dict.py ...
tesdat/tests/containers/test_iter.py ...
tesdat/tests/containers/test_list.py ..
tesdat/tests/containers/test_ordereddict.py .
tesdat/tests/formatters/test_csv.py ...
tesdat/tests/formatters/test_json.py F
tesdat/tests/formatters/test_pickle.py .
tesdat/tests/formatters/test_string.py F.
tesdat/tests/models/test_dict.py .....
tesdat/tests/models/test_list.py ......
tesdat/tests/patterns/test_choice.py ..
tesdat/tests/patterns/test_cycle.py ..
tesdat/tests/patterns/test_hashof.py .
tesdat/tests/patterns/test_increment.py ..
tesdat/tests/patterns/test_pickout.py ..
tesdat/tests/patterns/test_sequence.py ..
tesdat/tests/utils/test_datetime.py .......

==================================================== FAILURES ====================================================
________________________________________ TestJsonFormatter.test_stringify ________________________________________

self =

    def test_stringify(self):
        sf = self._getClass({'a': time(23, 59), 'b': ('2',)}, indent=None)
        self.assertEqual(
            sf.stringify(),
>           '{"a": "23:59:00", "b": ["2"]}'
        )
E       AssertionError: '{"b": ["2"], "a": "23:59:00"}' != '{"a": "23:59:00", "b": ["2"]}'
E       - {"b": ["2"], "a": "23:59:00"}
E       + {"a": "23:59:00", "b": ["2"]}

tesdat/tests/formatters/test_json.py:15: AssertionError
________________________________________ TestStringFormatter.test_string _________________________________________

self =

    def test_string(self):
        sf = self._getClass({'a': 1, 'b': ('2',)})
        self.assertEqual(
            sf.stringify(),
>           "{'a': 1, 'b': ('2',)}"
        )
E       AssertionError: "{'b': ('2',), 'a': 1}" != "{'a': 1, 'b': ('2',)}"
E       - {'b': ('2',), 'a': 1}
E       + {'a': 1, 'b': ('2',)}

tesdat/tests/formatters/test_string.py:23: AssertionError
====================================== 2 failed, 55 passed in 0.93 seconds =======================================
ERROR: InvocationError: '/home/vagrant/venv3.5/bin/py.test'
py34 inst-nodeps: /home/vagrant/tesdat/.tox/dist/tesdat-2.1.1.zip
py34 installed: tesdat==2.1.1
py34 runtests: PYTHONHASHSEED='1127406230'
py34 runtests: commands[0] | py.test
WARNING:test command found but not installed in testenv
  cmd: /home/vagrant/venv3.5/bin/py.test
  env: /home/vagrant/tesdat/.tox/py34
Maybe you forgot to specify a dependency? See also the whitelist_externals envconfig setting.
WARNING: Attempting to work in a virtualenv. If you encounter problems, please install IPython inside the virtualenv.
WARNING: IPython History requires SQLite, your history will not be saved
============================================== test session starts ===============================================
platform linux -- Python 3.5.1, pytest-2.9.1, py-1.4.31, pluggy-0.3.1
rootdir: /home/vagrant/tesdat, inifile:
plugins: ipdb-0.1.dev2, django-2.9.1
collected 57 items

tesdat/tests/test_api.py ....
tesdat/tests/test_example.py ...
tesdat/tests/test_index.py ..
tesdat/tests/containers/test_base.py ...
tesdat/tests/containers/test_dict.py ...
tesdat/tests/containers/test_iter.py ...
tesdat/tests/containers/test_list.py ..
tesdat/tests/containers/test_ordereddict.py .
tesdat/tests/formatters/test_csv.py ...
tesdat/tests/formatters/test_json.py F
tesdat/tests/formatters/test_pickle.py .
tesdat/tests/formatters/test_string.py F.
tesdat/tests/models/test_dict.py .....
tesdat/tests/models/test_list.py ......
tesdat/tests/patterns/test_choice.py ..
tesdat/tests/patterns/test_cycle.py ..
tesdat/tests/patterns/test_hashof.py .
tesdat/tests/patterns/test_increment.py ..
tesdat/tests/patterns/test_pickout.py ..
tesdat/tests/patterns/test_sequence.py ..
tesdat/tests/utils/test_datetime.py .......

==================================================== FAILURES ====================================================
________________________________________ TestJsonFormatter.test_stringify ________________________________________

self =

    def test_stringify(self):
        sf = self._getClass({'a': time(23, 59), 'b': ('2',)}, indent=None)
        self.assertEqual(
            sf.stringify(),
>           '{"a": "23:59:00", "b": ["2"]}'
        )
E       AssertionError: '{"b": ["2"], "a": "23:59:00"}' != '{"a": "23:59:00", "b": ["2"]}'
E       - {"b": ["2"], "a": "23:59:00"}
E       + {"a": "23:59:00", "b": ["2"]}

tesdat/tests/formatters/test_json.py:15: AssertionError
________________________________________ TestStringFormatter.test_string _________________________________________

self =

    def test_string(self):
        sf = self._getClass({'a': 1, 'b': ('2',)})
        self.assertEqual(
            sf.stringify(),
>           "{'a': 1, 'b': ('2',)}"
        )
E       AssertionError: "{'b': ('2',), 'a': 1}" != "{'a': 1, 'b': ('2',)}"
E       - {'b': ('2',), 'a': 1}
E       + {'a': 1, 'b': ('2',)}

tesdat/tests/formatters/test_string.py:23: AssertionError
====================================== 2 failed, 55 passed in 0.92 seconds =======================================
ERROR: InvocationError: '/home/vagrant/venv3.5/bin/py.test'
py35 inst-nodeps: /home/vagrant/tesdat/.tox/dist/tesdat-2.1.1.zip
py35 installed: tesdat==2.1.1
py35 runtests: PYTHONHASHSEED='1127406230'
py35 runtests: commands[0] | py.test
WARNING:test command found but not installed in testenv
  cmd: /home/vagrant/venv3.5/bin/py.test
  env: /home/vagrant/tesdat/.tox/py35
Maybe you forgot to specify a dependency? See also the whitelist_externals envconfig setting.
WARNING: Attempting to work in a virtualenv. If you encounter problems, please install IPython inside the virtualenv.
WARNING: IPython History requires SQLite, your history will not be saved
============================================== test session starts ===============================================
platform linux -- Python 3.5.1, pytest-2.9.1, py-1.4.31, pluggy-0.3.1
rootdir: /home/vagrant/tesdat, inifile:
plugins: ipdb-0.1.dev2, django-2.9.1
collected 57 items

tesdat/tests/test_api.py ....
tesdat/tests/test_example.py ...
tesdat/tests/test_index.py ..
tesdat/tests/containers/test_base.py ...
tesdat/tests/containers/test_dict.py ...
tesdat/tests/containers/test_iter.py ...
tesdat/tests/containers/test_list.py ..
tesdat/tests/containers/test_ordereddict.py .
tesdat/tests/formatters/test_csv.py ...
tesdat/tests/formatters/test_json.py F
tesdat/tests/formatters/test_pickle.py .
tesdat/tests/formatters/test_string.py F.
tesdat/tests/models/test_dict.py .....
tesdat/tests/models/test_list.py ......
tesdat/tests/patterns/test_choice.py ..
tesdat/tests/patterns/test_cycle.py ..
tesdat/tests/patterns/test_hashof.py .
tesdat/tests/patterns/test_increment.py ..
tesdat/tests/patterns/test_pickout.py ..
tesdat/tests/patterns/test_sequence.py ..
tesdat/tests/utils/test_datetime.py .......

==================================================== FAILURES ====================================================
________________________________________ TestJsonFormatter.test_stringify ________________________________________

self =

    def test_stringify(self):
        sf = self._getClass({'a': time(23, 59), 'b': ('2',)}, indent=None)
        self.assertEqual(
            sf.stringify(),
>           '{"a": "23:59:00", "b": ["2"]}'
        )
E       AssertionError: '{"b": ["2"], "a": "23:59:00"}' != '{"a": "23:59:00", "b": ["2"]}'
E       - {"b": ["2"], "a": "23:59:00"}
E       + {"a": "23:59:00", "b": ["2"]}

tesdat/tests/formatters/test_json.py:15: AssertionError
________________________________________ TestStringFormatter.test_string _________________________________________

self =

    def test_string(self):
        sf = self._getClass({'a': 1, 'b': ('2',)})
        self.assertEqual(
            sf.stringify(),
>           "{'a': 1, 'b': ('2',)}"
        )
E       AssertionError: "{'b': ('2',), 'a': 1}" != "{'a': 1, 'b': ('2',)}"
E       - {'b': ('2',), 'a': 1}
E       + {'a': 1, 'b': ('2',)}

tesdat/tests/formatters/test_string.py:23: AssertionError
====================================== 2 failed, 55 passed in 0.99 seconds =======================================
ERROR: InvocationError: '/home/vagrant/venv3.5/bin/py.test'
flake8 recreate: /home/vagrant/tesdat/.tox/flake8
flake8 installdeps: flake8, flake8-blind-except, flake8-docstrings, flake8-import-order, mccabe, radon
flake8 inst: /home/vagrant/tesdat/.tox/dist/tesdat-2.1.1.zip
flake8 installed: colorama==0.3.7,flake8==2.5.4,flake8-blind-except==0.1.0,flake8-docstrings==0.2.5,flake8-import-order==0.7,mando==0.3.3,mccabe==0.4.0,pep257==0.7.0,pep8==1.7.0,pyflakes==1.0.0,radon==1.3.2,tesdat==2.1.1
flake8 runtests: PYTHONHASHSEED='1127406230'
flake8 runtests: commands[0] | flake8 .
./setup.py:1:1: D100 Missing docstring in public module
./setup.py:4:1: I101 Imported names are in the wrong order. Should be find_packages, setup
./setup.py:5:1: I101 Imported names are in the wrong order. Should be __author__, __author_email__, __license__, __version__
./setup.py:5:1: I201 Missing newline before sections or imports.
./tesdat/__init__.py:1:1: D200 One-line docstring should fit on one line with quotes
./tesdat/__init__.py:11:1: E402 module level import not at top of file
./tesdat/__init__.py:12:1: E402 module level import not at top of file
./tesdat/__init__.py:12:1: I100 Import statements are in the wrong order. from .containers.dict should be before from .containers.list
./tesdat/__init__.py:13:1: E402 module level import not at top of file
./tesdat/__init__.py:15:1: E402 module level import not at top of file
./tesdat/__init__.py:16:1: E402 module level import not at top of file
./tesdat/__init__.py:16:1: I100 Import statements are in the wrong order. from .models.dict should be before from .models.list
./tesdat/__init__.py:18:1: E402 module level import not at top of file
./tesdat/__init__.py:19:1: E402 module level import not at top of file
./tesdat/__init__.py:20:1: E402 module level import not at top of file
./tesdat/__init__.py:20:1: I100 Import statements are in the wrong order. from .patterns.cycle should be before from .patterns.pickout
./tesdat/__init__.py:21:1: E402 module level import not at top of file
./tesdat/__init__.py:22:1: E402 module level import not at top of file
./tesdat/__init__.py:23:1: E402 module level import not at top of file
./tesdat/__init__.py:26:1: E402 module level import not at top of file
./tesdat/__init__.py:26:1: I100 Import statements are in the wrong order. from .formatters.string should be before from .patterns.sequence
./tesdat/__init__.py:27:1: E402 module level import not at top of file
./tesdat/__init__.py:27:1: I100 Import statements are in the wrong order. from .formatters.json should be before from .formatters.string
./tesdat/__init__.py:28:1: E402 module level import not at top of file
./tesdat/__init__.py:29:1: E402 module level import not at top of file
./tesdat/__init__.py:29:1: I100 Import statements are in the wrong order. from .formatters.csv should be before from .formatters.pickle
./tesdat/__init__.py:31:1: E402 module level import not at top of file
./tesdat/__init__.py:31:1: F401 'BLANK' imported but unused
./tesdat/__init__.py:31:1: F401 'ESCAPE' imported but unused
./tesdat/__init__.py:31:1: I100 Import statements are in the wrong order. from .api.special should be before from .formatters.csv
./tesdat/__init__.py:76:1: D205 1 blank line required between summary line and description
./tesdat/__init__.py:76:1: D400 First line should end with a period
./tesdat/__init__.py:88:9: F821 undefined name 'containers'
./tesdat/__init__.py:88:21: F821 undefined name 'models'
./tesdat/__init__.py:88:29: F821 undefined name 'patterns'
./tesdat/__init__.py:88:39: F821 undefined name 'formatters'
./tesdat/__init__.py:88:51: F821 undefined name 'exceptions'
./tesdat/__init__.py:88:63: F821 undefined name 'api'
./tesdat/compat.py:1:1: D100 Missing docstring in public module
./tesdat/compat.py:30:9: F401 'StringIO' imported but unused
./tesdat/exceptions.py:1:1: D100 Missing docstring in public module
./tesdat/exceptions.py:3:1: D101 Missing docstring in public class
./tesdat/exceptions.py:3:1: E302 expected 2 blank lines, found 1
./tesdat/exceptions.py:7:1: D101 Missing docstring in public class
./tesdat/exceptions.py:11:1: D101 Missing docstring in public class
./tesdat/exceptions.py:15:1: D101 Missing docstring in public class
./tesdat/api/__init__.py:1:1: D104 Missing docstring in public package
./tesdat/api/display.py:1:1: D100 Missing docstring in public module
./tesdat/api/display.py:5:1: D103 Missing docstring in public function
./tesdat/api/display.py:12:1: D103 Missing docstring in public function
./tesdat/api/display.py:22:1: W391 blank line at end of file
./tesdat/api/render.py:1:1: D100 Missing docstring in public module
./tesdat/api/render.py:7:1: I100 Import statements are in the wrong order. from .. should be before from ..patterns.base
./tesdat/api/render.py:9:1: E302 expected 2 blank lines, found 1
./tesdat/api/render.py:15:1: D205 1 blank line required between summary line and description
./tesdat/api/render.py:15:1: D400 First line should end with a period
./tesdat/api/render.py:37:80: E501 line too long (80 > 79 characters)
./tesdat/api/special.py:1:1: D100 Missing docstring in public module
./tesdat/api/special.py:4:1: D204 1 blank line required after class docstring
./tesdat/api/special.py:4:1: D400 First line should end with a period
./tesdat/api/special.py:6:1: D102 Missing docstring in public method
./tesdat/api/special.py:9:1: D105 Missing docstring in magic method
./tesdat/containers/__init__.py:1:1: D104 Missing docstring in public package
./tesdat/containers/base.py:1:1: D100 Missing docstring in public module
./tesdat/containers/base.py:9:1: D205 1 blank line required between summary line and description
./tesdat/containers/base.py:9:1: D400 First line should end with a period
./tesdat/containers/base.py:23:1: D200 One-line docstring should fit on one line with quotes
./tesdat/containers/base.py:23:1: D204 1 blank line required after class docstring
./tesdat/containers/base.py:26:1: D205 1 blank line required between summary line and description
./tesdat/containers/base.py:26:1: D400 First line should end with a period
./tesdat/containers/base.py:26:80: E501 line too long (90 > 79 characters)
./tesdat/containers/base.py:28:80: E501 line too long (87 > 79 characters)
./tesdat/containers/base.py:41:1: D102 Missing docstring in public method
./tesdat/containers/base.py:62:80: E501 line too long (81 > 79 characters)
./tesdat/containers/base.py:65:1: D102 Missing docstring in public method
./tesdat/containers/dict.py:1:1: D100 Missing docstring in public module
./tesdat/containers/dict.py:8:1: D101 Missing docstring in public class
./tesdat/containers/iter.py:1:1: D100 Missing docstring in public module
./tesdat/containers/iter.py:8:1: D200 One-line docstring should fit on one line with quotes
./tesdat/containers/iter.py:8:1: D204 1 blank line required after class docstring
./tesdat/containers/iter.py:8:1: D400 First line should end with a period
./tesdat/containers/iter.py:14:1: D105 Missing docstring in magic method
./tesdat/containers/iter.py:27:1: D102 Missing docstring in public method
./tesdat/containers/iter.py:30:1: D102 Missing docstring in public method
./tesdat/containers/iter.py:41:1: D105 Missing docstring in magic method
./tesdat/containers/list.py:1:1: D100 Missing docstring in public module
./tesdat/containers/list.py:8:1: D101 Missing docstring in public class
./tesdat/containers/list.py:12:80: E501 line too long (125 > 79 characters)
./tesdat/containers/list.py:28:1: D200 One-line docstring should fit on one line with quotes
./tesdat/containers/list.py:28:1: D400 First line should end with a period
./tesdat/containers/ordereddict.py:1:1: D100 Missing docstring in public module
./tesdat/containers/ordereddict.py:4:1: I100 Import statements are in the wrong order. from collections should be  before from .dict
./tesdat/containers/ordereddict.py:4:1: I201 Missing newline before sections or imports.
./tesdat/containers/ordereddict.py:7:1: D101 Missing docstring in public class
./tesdat/containers/ordereddict.py:8:1: D102 Missing docstring in public method
./tesdat/formatters/__init__.py:1:1: D104 Missing docstring in public package
./tesdat/formatters/base.py:1:1: D100 Missing docstring in public module
./tesdat/formatters/base.py:3:1: I100 Import statements are in the wrong order. import codecs should be before import os
./tesdat/formatters/base.py:8:1: D101 Missing docstring in public class
./tesdat/formatters/base.py:22:1: D102 Missing docstring in public method
./tesdat/formatters/csv.py:1:1: D100 Missing docstring in public module
./tesdat/formatters/csv.py:4:1: F401 'io' imported but unused
./tesdat/formatters/csv.py:5:1: F401 'os' imported but unused
./tesdat/formatters/csv.py:6:1: I100 Import statements are in the wrong order. import csv should be before import os
./tesdat/formatters/csv.py:9:1: F401 'OutputFileAlreadyExists' imported but unused
./tesdat/formatters/csv.py:9:1: I101 Imported names are in the wrong order. Should be CsvFormatterFieldsTypeInvalid, OutputFileAlreadyExists
./tesdat/formatters/csv.py:14:1: I100 Import statements are in the wrong order. from .. should be before from ..exceptions
./tesdat/formatters/csv.py:17:1: D101 Missing docstring in public class
./tesdat/formatters/csv.py:18:1: D102 Missing docstring in public method
./tesdat/formatters/csv.py:57:1: D102 Missing docstring in public method
./tesdat/formatters/json.py:1:1: D100 Missing docstring in public module
./tesdat/formatters/json.py:3:1: I201 Missing newline before sections or imports.
./tesdat/formatters/json.py:4:1: I101 Imported names are in the wrong order. Should be date, datetime, time
./tesdat/formatters/json.py:5:1: I201 Missing newline before sections or imports.
./tesdat/formatters/json.py:8:1: D200 One-line docstring should fit on one line with quotes
./tesdat/formatters/json.py:8:1: D204 1 blank line required after class docstring
./tesdat/formatters/json.py:18:1: D102 Missing docstring in public method
./tesdat/formatters/json.py:26:1: D101 Missing docstring in public class
./tesdat/formatters/json.py:27:1: D102 Missing docstring in public method
./tesdat/formatters/json.py:33:1: D102 Missing docstring in public method
./tesdat/formatters/pickle.py:1:1: D100 Missing docstring in public module
./tesdat/formatters/pickle.py:3:1: I201 Missing newline before sections or imports.
./tesdat/formatters/pickle.py:4:1: I201 Missing newline before sections or imports.
./tesdat/formatters/pickle.py:7:1: D101 Missing docstring in public class
./tesdat/formatters/pickle.py:8:1: D102 Missing docstring in public method
./tesdat/formatters/pickle.py:12:1: D102 Missing docstring in public method
./tesdat/formatters/string.py:1:1: D100 Missing docstring in public module
./tesdat/formatters/string.py:3:1: I201 Missing newline before sections or imports.
./tesdat/formatters/string.py:6:1: D101 Missing docstring in public class
./tesdat/formatters/string.py:7:1: D102 Missing docstring in public method
./tesdat/formatters/string.py:10:1: D102 Missing docstring in public method
./tesdat/models/__init__.py:1:1: D104 Missing docstring in public package
./tesdat/models/base.py:1:1: D100 Missing docstring in public module
./tesdat/models/base.py:4:1: D101 Missing docstring in public class
./tesdat/models/dict.py:1:1: D100 Missing docstring in public module
./tesdat/models/dict.py:6:1: D200 One-line docstring should fit on one line with quotes
./tesdat/models/dict.py:6:1: D204 1 blank line required after class docstring
./tesdat/models/dict.py:26:80: E501 line too long (83 > 79 characters)
./tesdat/models/dict.py:39:80: E501 line too long (88 > 79 characters)
./tesdat/models/dict.py:48:13: E265 block comment should start with '# '
./tesdat/models/list.py:1:1: D100 Missing docstring in public module
./tesdat/models/list.py:6:1: D200 One-line docstring should fit on one line with quotes
./tesdat/models/list.py:6:1: D204 1 blank line required after class docstring
./tesdat/models/list.py:40:1: D205 1 blank line required between summary line and description
./tesdat/models/list.py:44:80: E501 line too long (88 > 79 characters)
./tesdat/models/ordereddict.py:1:1: D100 Missing docstring in public module
./tesdat/models/ordereddict.py:3:1: I201 Missing newline before sections or imports.
./tesdat/models/ordereddict.py:6:1: D200 One-line docstring should fit on one line with quotes
./tesdat/models/ordereddict.py:6:1: D204 1 blank line required after class docstring
./tesdat/models/ordereddict.py:11:1: D102 Missing docstring in public method
./tesdat/patterns/__init__.py:1:1: D104 Missing docstring in public package
./tesdat/patterns/__init__.py:2:1: W391 blank line at end of file
./tesdat/patterns/base.py:1:1: D100 Missing docstring in public module
./tesdat/patterns/base.py:4:1: D101 Missing docstring in public class
./tesdat/patterns/choice.py:1:1: D100 Missing docstring in public module
./tesdat/patterns/choice.py:3:1: I201 Missing newline before sections or imports.
./tesdat/patterns/choice.py:7:1: D200 One-line docstring should fit on one line with quotes
./tesdat/patterns/choice.py:7:1: D204 1 blank line required after class docstring
./tesdat/patterns/choice.py:10:1: D205 1 blank line required between summary line and description
./tesdat/patterns/choice.py:10:1: D400 First line should end with a period
./tesdat/patterns/choice.py:24:1: D105 Missing docstring in magic method
./tesdat/patterns/choice.py:28:1: D400 First line should end with a period
./tesdat/patterns/cycle.py:1:1: D100 Missing docstring in public module
./tesdat/patterns/cycle.py:6:1: D200 One-line docstring should fit on one line with quotes
./tesdat/patterns/cycle.py:6:1: D204 1 blank line required after class docstring
./tesdat/patterns/cycle.py:9:1: D205 1 blank line required between summary line and description
./tesdat/patterns/cycle.py:9:1: D400 First line should end with a period
./tesdat/patterns/cycle.py:11:80: E501 line too long (95 > 79 characters)
./tesdat/patterns/cycle.py:14:80: E501 line too long (80 > 79 characters)
./tesdat/patterns/cycle.py:31:1: D105 Missing docstring in magic method
./tesdat/patterns/hashof.py:1:1: D100 Missing docstring in public module
./tesdat/patterns/hashof.py:10:1: D103 Missing docstring in public function
./tesdat/patterns/hashof.py:20:1: D200 One-line docstring should fit on one line with quotes
./tesdat/patterns/hashof.py:20:1: D204 1 blank line required after class docstring
./tesdat/patterns/hashof.py:23:1: D205 1 blank line required between summary line and description
./tesdat/patterns/hashof.py:23:80: E501 line too long (84 > 79 characters)
./tesdat/patterns/hashof.py:26:80: E501 line too long (92 > 79 characters)
./tesdat/patterns/hashof.py:28:80: E501 line too long (80 > 79 characters)
./tesdat/patterns/hashof.py:37:1: D105 Missing docstring in magic method
./tesdat/patterns/increment.py:1:1: D100 Missing docstring in public module
./tesdat/patterns/increment.py:7:1: D200 One-line docstring should fit on one line with quotes
./tesdat/patterns/increment.py:7:1: D204 1 blank line required after class docstring
./tesdat/patterns/increment.py:10:1: D205 1 blank line required between summary line and description
./tesdat/patterns/increment.py:14:80: E501 line too long (80 > 79 characters)
./tesdat/patterns/increment.py:21:1: D105 Missing docstring in magic method
./tesdat/patterns/pickout.py:1:1: D100 Missing docstring in public module
./tesdat/patterns/pickout.py:8:1: D200 One-line docstring should fit on one line with quotes
./tesdat/patterns/pickout.py:8:1: D204 1 blank line required after class docstring
./tesdat/patterns/pickout.py:11:1: D205 1 blank line required between summary line and description
./tesdat/patterns/pickout.py:11:1: D400 First line should end with a period
./tesdat/patterns/pickout.py:13:80: E501 line too long (93 > 79 characters)
./tesdat/patterns/pickout.py:15:80: E501 line too long (80 > 79 characters)
./tesdat/patterns/pickout.py:27:1: D105 Missing docstring in magic method
./tesdat/patterns/pickout.py:31:1: D400 First line should end with a period
./tesdat/patterns/sequence.py:1:1: D100 Missing docstring in public module
./tesdat/patterns/sequence.py:7:1: D200 One-line docstring should fit on one line with quotes
./tesdat/patterns/sequence.py:7:1: D204 1 blank line required after class docstring
./tesdat/patterns/sequence.py:10:1: D205 1 blank line required between summary line and description
./tesdat/patterns/sequence.py:10:1: D400 First line should end with a period
./tesdat/patterns/sequence.py:12:80: E501 line too long (95 > 79 characters)
./tesdat/patterns/sequence.py:15:80: E501 line too long (80 > 79 characters)
./tesdat/patterns/sequence.py:32:1: D105 Missing docstring in magic method
./tesdat/patterns/sequence.py:36:1: D400 First line should end with a period
./tesdat/tests/__init__.py:1:1: D104 Missing docstring in public package
./tesdat/tests/test_api.py:1:1: D100 Missing docstring in public module
./tesdat/tests/test_api.py:5:1: D101 Missing docstring in public class
./tesdat/tests/test_api.py:10:1: D102 Missing docstring in public method
./tesdat/tests/test_api.py:17:1: D102 Missing docstring in public method
./tesdat/tests/test_api.py:20:1: D102 Missing docstring in public method
./tesdat/tests/test_api.py:23:80: E501 line too long (84 > 79 characters)
./tesdat/tests/test_api.py:25:1: D102 Missing docstring in public method
./tesdat/tests/test_example.py:1:1: D100 Missing docstring in public module
./tesdat/tests/test_example.py:5:1: D101 Missing docstring in public class
./tesdat/tests/test_example.py:6:1: D102 Missing docstring in public method
./tesdat/tests/test_example.py:25:1: D102 Missing docstring in public method
./tesdat/tests/test_example.py:43:1: D102 Missing docstring in public method
./tesdat/tests/test_example.py:54:80: E501 line too long (81 > 79 characters)
./tesdat/tests/test_index.py:1:1: D100 Missing docstring in public module
./tesdat/tests/test_index.py:5:1: D101 Missing docstring in public class
./tesdat/tests/test_index.py:10:1: D102 Missing docstring in public method
./tesdat/tests/test_index.py:14:1: D102 Missing docstring in public method
./tesdat/tests/containers/__init__.py:1:1: D104 Missing docstring in public package
./tesdat/tests/containers/test_base.py:1:1: D100 Missing docstring in public module
./tesdat/tests/containers/test_base.py:5:1: D101 Missing docstring in public class
./tesdat/tests/containers/test_base.py:10:1: D102 Missing docstring in public method
./tesdat/tests/containers/test_base.py:15:1: D102 Missing docstring in public method
./tesdat/tests/containers/test_base.py:23:1: D101 Missing docstring in public class
./tesdat/tests/containers/test_base.py:24:1: D102 Missing docstring in public method
./tesdat/tests/containers/test_dict.py:1:1: D100 Missing docstring in public module
./tesdat/tests/containers/test_dict.py:5:1: D101 Missing docstring in public class
./tesdat/tests/containers/test_dict.py:10:1: D102 Missing docstring in public method
./tesdat/tests/containers/test_dict.py:25:1: D102 Missing docstring in public method
./tesdat/tests/containers/test_dict.py:40:1: D102 Missing docstring in public method
./tesdat/tests/containers/test_iter.py:1:1: D100 Missing docstring in public module
./tesdat/tests/containers/test_iter.py:5:1: D101 Missing docstring in public class
./tesdat/tests/containers/test_iter.py:10:1: D102 Missing docstring in public method
./tesdat/tests/containers/test_iter.py:14:1: D102 Missing docstring in public method
./tesdat/tests/containers/test_iter.py:25:1: D102 Missing docstring in public method
./tesdat/tests/containers/test_list.py:1:1: D100 Missing docstring in public module
./tesdat/tests/containers/test_list.py:5:1: D101 Missing docstring in public class
./tesdat/tests/containers/test_list.py:10:1: D102 Missing docstring in public method
./tesdat/tests/containers/test_list.py:25:1: D102 Missing docstring in public method
./tesdat/tests/containers/test_ordereddict.py:1:1: D100 Missing docstring in public module
./tesdat/tests/containers/test_ordereddict.py:5:1: D101 Missing docstring in public class
./tesdat/tests/containers/test_ordereddict.py:10:1: D102 Missing docstring in public method
./tesdat/tests/formatters/__init__.py:1:1: D104 Missing docstring in public package
./tesdat/tests/formatters/test_base.py:1:1: D100 Missing docstring in public module
./tesdat/tests/formatters/test_csv.py:1:1: D100 Missing docstring in public module
./tesdat/tests/formatters/test_csv.py:3:1: F401 'time' imported but unused
./tesdat/tests/formatters/test_csv.py:7:1: D101 Missing docstring in public class
./tesdat/tests/formatters/test_csv.py:10:1: D102 Missing docstring in public method
./tesdat/tests/formatters/test_csv.py:20:1: D102 Missing docstring in public method
./tesdat/tests/formatters/test_csv.py:33:1: D102 Missing docstring in public method
./tesdat/tests/formatters/test_csv.py:46:1: D102 Missing docstring in public method
./tesdat/tests/formatters/test_json.py:1:1: D100 Missing docstring in public module
./tesdat/tests/formatters/test_json.py:6:1: D101 Missing docstring in public class
./tesdat/tests/formatters/test_json.py:11:1: D102 Missing docstring in public method
./tesdat/tests/formatters/test_pickle.py:1:1: D100 Missing docstring in public module
./tesdat/tests/formatters/test_pickle.py:7:1: D101 Missing docstring in public class
./tesdat/tests/formatters/test_pickle.py:10:1: D102 Missing docstring in public method
./tesdat/tests/formatters/test_pickle.py:20:1: D102 Missing docstring in public method
./tesdat/tests/formatters/test_string.py:1:1: D100 Missing docstring in public module
./tesdat/tests/formatters/test_string.py:6:1: D101 Missing docstring in public class
./tesdat/tests/formatters/test_string.py:9:1: D102 Missing docstring in public method
./tesdat/tests/formatters/test_string.py:19:1: D102 Missing docstring in public method
./tesdat/tests/formatters/test_string.py:26:1: D102 Missing docstring in public method
./tesdat/tests/models/__init__.py:1:1: D104 Missing docstring in public package
./tesdat/tests/models/test_dict.py:1:1: D100 Missing docstring in public module
./tesdat/tests/models/test_dict.py:5:1: D103 Missing docstring in public function
./tesdat/tests/models/test_dict.py:9:1: D101 Missing docstring in public class
./tesdat/tests/models/test_dict.py:14:1: D102 Missing docstring in public method
./tesdat/tests/models/test_dict.py:21:1: D102 Missing docstring in public method
./tesdat/tests/models/test_dict.py:42:1: D102 Missing docstring in public method
./tesdat/tests/models/test_dict.py:56:1: D102 Missing docstring in public method
./tesdat/tests/models/test_dict.py:76:1: D102 Missing docstring in public method
./tesdat/tests/models/test_list.py:1:1: D100 Missing docstring in public module
./tesdat/tests/models/test_list.py:6:1: D103 Missing docstring in public function
./tesdat/tests/models/test_list.py:10:1: D101 Missing docstring in public class
./tesdat/tests/models/test_list.py:15:1: D102 Missing docstring in public method
./tesdat/tests/models/test_list.py:20:1: D102 Missing docstring in public method
./tesdat/tests/models/test_list.py:25:1: D102 Missing docstring in public method
./tesdat/tests/models/test_list.py:46:1: D102 Missing docstring in public method
./tesdat/tests/models/test_list.py:60:1: D102 Missing docstring in public method
./tesdat/tests/models/test_list.py:77:1: D102 Missing docstring in public method
./tesdat/tests/patterns/__init__.py:1:1: D104 Missing docstring in public package
./tesdat/tests/patterns/test_choice.py:1:1: D100 Missing docstring in public module
./tesdat/tests/patterns/test_choice.py:5:1: D101 Missing docstring in public class
./tesdat/tests/patterns/test_choice.py:10:1: D102 Missing docstring in public method
./tesdat/tests/patterns/test_choice.py:17:1: D102 Missing docstring in public method
./tesdat/tests/patterns/test_cycle.py:1:1: D100 Missing docstring in public module
./tesdat/tests/patterns/test_cycle.py:5:1: D101 Missing docstring in public class
./tesdat/tests/patterns/test_cycle.py:10:1: D102 Missing docstring in public method
./tesdat/tests/patterns/test_cycle.py:17:1: D102 Missing docstring in public method
./tesdat/tests/patterns/test_hashof.py:1:1: D100 Missing docstring in public module
./tesdat/tests/patterns/test_hashof.py:11:1: D101 Missing docstring in public class
./tesdat/tests/patterns/test_hashof.py:16:1: D102 Missing docstring in public method
./tesdat/tests/patterns/test_increment.py:1:1: D100 Missing docstring in public module
./tesdat/tests/patterns/test_increment.py:6:1: D101 Missing docstring in public class
./tesdat/tests/patterns/test_increment.py:11:1: D102 Missing docstring in public method
./tesdat/tests/patterns/test_increment.py:18:1: D102 Missing docstring in public method
./tesdat/tests/patterns/test_pickout.py:1:1: D100 Missing docstring in public module
./tesdat/tests/patterns/test_pickout.py:5:1: D101 Missing docstring in public class
./tesdat/tests/patterns/test_pickout.py:10:1: D102 Missing docstring in public method
./tesdat/tests/patterns/test_pickout.py:19:1: D102 Missing docstring in public method
./tesdat/tests/patterns/test_sequence.py:1:1: D100 Missing docstring in public module
./tesdat/tests/patterns/test_sequence.py:5:1: D101 Missing docstring in public class
./tesdat/tests/patterns/test_sequence.py:10:1: D102 Missing docstring in public method
./tesdat/tests/patterns/test_sequence.py:19:1: D102 Missing docstring in public method
./tesdat/tests/utils/__init__.py:1:1: D104 Missing docstring in public package
./tesdat/tests/utils/test_datetime.py:1:1: D100 Missing docstring in public module
./tesdat/tests/utils/test_datetime.py:3:1: I100 Import statements are in the wrong order. from datetime should be before from unittest
./tesdat/tests/utils/test_datetime.py:9:1: D101 Missing docstring in public class
./tesdat/tests/utils/test_datetime.py:14:1: D102 Missing docstring in public method
./tesdat/tests/utils/test_datetime.py:20:1: D102 Missing docstring in public method
./tesdat/tests/utils/test_datetime.py:30:1: D102 Missing docstring in public method
./tesdat/tests/utils/test_datetime.py:43:1: D102 Missing docstring in public method
./tesdat/tests/utils/test_datetime.py:59:1: D101 Missing docstring in public class
./tesdat/tests/utils/test_datetime.py:64:1: D102 Missing docstring in public method
./tesdat/tests/utils/test_datetime.py:73:80: E501 line too long (85 > 79 characters)
./tesdat/tests/utils/test_datetime.py:76:1: D101 Missing docstring in public class
./tesdat/tests/utils/test_datetime.py:81:1: D102 Missing docstring in public method
./tesdat/tests/utils/test_datetime.py:85:1: D102 Missing docstring in public method
./tesdat/utils/__init__.py:1:1: D104 Missing docstring in public package
./tesdat/utils/datetime.py:1:1: D100 Missing docstring in public module
./tesdat/utils/datetime.py:6:1: I101 Imported names are in the wrong order. Should be date, datetime, time, timedelta
./tesdat/utils/datetime.py:20:1: E402 module level import not at top of file
./tesdat/utils/datetime.py:63:1: D400 First line should end with a period
./tesdat/utils/datetime.py:87:1: D400 First line should end with a period
ERROR: InvocationError: '/home/vagrant/tesdat/.tox/flake8/bin/flake8 .'
____________________________________________________ summary _____________________________________________________
ERROR:   py27: commands failed
ERROR:   py33: commands failed
ERROR:   py34: commands failed
ERROR:   py35: commands failed
ERROR:   flake8: commands failed

ってうわー、すごいエラー(警告)が。

実はエラーに関しては前から気付いていて、辞書の値が特定の順番になることを前提にテストを書いていたため、エラーになっています。

また脱線しますが、これは実行時に決まる PYTHONHASHSEED という環境変数に左右されます。

この場合は PYTHONHASHSEED='115669612' であればテストは通るのです。

(venv3.5) [vagrant@localhost tesdat]$ PYTHONHASHSEED='115669612' tox
========================================================================================== 中略 ==========================================================================================
___________________________________________________________ summary ___________________________________________________________
py27: commands succeeded
py33: commands succeeded
py34: commands succeeded
py35: commands succeeded
ERROR:   flake8: commands failed

とりあえず、テストを直します。興味ないかもしれませんが、こんな感じ。

diff --git a/tesdat/tests/formatters/test_json.py b/tesdat/tests/formatters/test_json.py
index 19f2542..796672b 100644
--- a/tesdat/tests/formatters/test_json.py
+++ b/tesdat/tests/formatters/test_json.py
@@ -1,4 +1,5 @@
 # coding: utf-8
+import json
 from datetime import time
 from unittest import TestCase

@@ -11,8 +12,8 @@ class TestJsonFormatter(TestCase):
     def test_stringify(self):
         sf = self._getClass({'a': time(23, 59), 'b': ('2',)}, indent=None)
         self.assertEqual(
-            sf.stringify(),
-            '{"a": "23:59:00", "b": ["2"]}'
+            json.loads(sf.stringify()),
+            {"a": "23:59:00", "b": ["2"]}
         )

         sf = self._getClass({list}, indent=None)
diff --git a/tesdat/tests/formatters/test_string.py b/tesdat/tests/formatters/test_string.py
index 372b6c7..94c025c 100644
--- a/tesdat/tests/formatters/test_string.py
+++ b/tesdat/tests/formatters/test_string.py
@@ -19,8 +19,8 @@ class TestStringFormatter(TestCase):
     def test_string(self):
         sf = self._getClass({'a': 1, 'b': ('2',)})
         self.assertEqual(
-            sf.stringify(),
-            "{'a': 1, 'b': ('2',)}"
+            eval(sf.stringify()),
+            {'a': 1, 'b': ('2',)}
         )

先ほどは失敗した PYTHONHASHSEED で通ることを確認しました。

(venv3.5) [vagrant@localhost tesdat]$ $ PYTHONHASHSEED='1127406230' tox
========================================================================================== 中略 ==========================================================================================
___________________________________________________________ summary ___________________________________________________________
py27: commands succeeded
py33: commands succeeded
py34: commands succeeded
py35: commands succeeded
ERROR:   flake8: commands failed

ちなみに PYTHONHASHSEED に限らず環境変数を固定したい場合、tox.iniの testenv セクションに setenv を指定します。

[testenv]
setenv =
  PYTHONHASHSEED = 1127406230

flake8

はい、 flake8 に手を付けます。ここは少々脱線気味なので読み飛ばしてもいいです。

この中には無視したいものもあるはずです。吟味したうえで要らないものは除外設定しましょう。除外設定は tox.ini の flake8 セクションに書きます。

それぞれ何を言われているのでしょうか。さすがに全てを洗い出すのはしんどいので簡単にまとめました。

CODE	対応	説明	要約
I100	無視	Import statements are in the wrong order.	importするモジュールの順序が不正。ただどうすればいいのかはよくわからない。
I101	無視	Imported names are in the wrong order. Should be xxx, yyy, zzz	import順がアルファベット順になっているべき。
I201	直す	Missing newline before sections or imports.	importする対象のモジュール種類によって空行を入れるべき
E226	直す	missing whitespace around arithmetic operator	演算子の前後に空行が必要。
E265	直す	block comment should start with '# '	コメント記号 `#` と文字列との間に空白が1つ必要。
E302	直す	expected 2 blank lines, found x	関数やクラスの定義は2行空けるべき。
E402	直す	module level import not at top of file	これは `__init__.py` で発生してる模様なので、このファイルをflake8の対象から除外。
E501	直す	line too long (xx > 79 characters)	よく見る1行当たりの文字数オーバー。個人で開発してるのにこんなこと気にしたくないです。とはいえ長すぎるのも考え物なので倍の160文字にします。
F401	直す	imported but unused	importされてるけど使われてない。本来であれば気にするんですが、このライブラリでは定義されていてユーザに使われる目的の変数もあるのです。なので、 `__init__.py` をflake8の対象から除外することで対応する。
F821	直す	undefined name 'xxx'	変数が定義されていない。これは `__init__.py` で発生してる模様なので、flake8の対象から除外することで対応する。
D100	無視	Missing docstring in public module	公開モジュールにdocstringで説明がない。ちなみにD開始はdocstring関連ですワンピースは関係ない。
D101	無視	Missing docstring in public class	公開クラスにdocstringがない。(以下略
D103	無視	Missing docstring in public function	公開メソッドに(以下略
D104	無視	Missing docstring in public package	公開パッケージに(以下略
D105	無視	Missing docstring in magic method	マジックメソッドに(以下略
D200	直す	One-line docstring should fit on one line with quotes	一行のdocstringは改行なしで表示しよう。
D204	直す	1 blank line required after class docstring	クラスレベルのdocstingの後には1行空行が必要。
D205	直す	1 blank line required between summary line and description	docstringの前後に不要な改行がある。見やすいように改行を入れた改行がよくなかったらしい。
D210	直す	No whitespaces allowed surrounding docstring text	docstringの前後には空白がいらない。
D400	無視	First line should end with a period	docstringの先頭行の末尾は .(ドット) で終わってないといけない
W391	直す	blank line at end of file	ファイルの末尾に余計な空行がある。

こんな感じで対応します。反映させた tox.ini が以下です。

[flake8]
ignore = D100, D101, D102, D103, D104, D105, D400, I100, I101
exclude = tests/*, tesdat/__init__.py, tesdat/compat.py
max-line-length = 160

もちろんコードも直しました。

.toxディレクトリ

一度toxを実行すると、 .tox/ 配下にenvlist毎の設定を作成し、次回以降は再利用されます。

(venv3.5) [vagrant@localhost tesdat]$ ls .tox/
dist  flake8  log  py27  py33  py34  py35

一旦初期化したい場合は「-r」「--recreate」オプションを指定しましょう。たまに前の実行結果に引きずられてテストが落ちたりすることがあるので、ハマったら一度試してみましょう。

カバレッジとる

ついでなのでカバレッジを取りましょう。 tox.iniを以下のように編集します。

[testenv]
commands =
  py.test --cov
deps =
  pytest-cov

pytest-cov をインストールし、--covオプションを指定します。これにより、実行時にカバレッジが取得されます。

$ coverage html を実行すると htmlcov/ に HTMLファイルが作成されるのでお好みで実行してください。

で、これだと検査したくないファイルまで対象となってしまうため、 coveragerc によって除外します。

[run]
omit = .tox/*, setup.py, tesdat/tests/*

あとリポジトリに入らないように .gitignore に以下を追加します。

.coverage
coverage.xml
htmlcov/*

実行してみましょう。

(venv3.5) [vagrant@localhost tesdat]$ tox -r
GLOB sdist-make: /home/vagrant/tesdat/setup.py
py27 recreate: /home/vagrant/tesdat/.tox/py27
py27 installdeps: pytest-cov
py27 inst: /home/vagrant/tesdat/.tox/dist/tesdat-2.1.1.zip
py27 installed: coverage==4.0.3,py==1.4.31,pytest==2.9.1,pytest-cov==2.2.1,tesdat==2.1.1
py27 runtests: PYTHONHASHSEED='2232003898'
py27 runtests: commands[0] | py.test --cov
================================================= test session starts =================================================
platform linux2 -- Python 2.7.10, pytest-2.9.1, py-1.4.31, pluggy-0.3.1
rootdir: /home/vagrant/tesdat, inifile:
plugins: cov-2.2.1
collected 58 items

tests-require.txt s
tesdat/tests/test_api.py ....
tesdat/tests/test_example.py ...
tesdat/tests/test_index.py ..
tesdat/tests/containers/test_base.py ...
tesdat/tests/containers/test_dict.py ...
tesdat/tests/containers/test_iter.py ...
tesdat/tests/containers/test_list.py ..
tesdat/tests/containers/test_ordereddict.py .
tesdat/tests/formatters/test_csv.py ...
tesdat/tests/formatters/test_json.py .
tesdat/tests/formatters/test_pickle.py .
tesdat/tests/formatters/test_string.py ..
tesdat/tests/models/test_dict.py .....
tesdat/tests/models/test_list.py ......
tesdat/tests/patterns/test_choice.py ..
tesdat/tests/patterns/test_cycle.py ..
tesdat/tests/patterns/test_hashof.py .
tesdat/tests/patterns/test_increment.py ..
tesdat/tests/patterns/test_pickout.py ..
tesdat/tests/patterns/test_sequence.py ..
tesdat/tests/utils/test_datetime.py .......
---------------------------------- coverage: platform linux2, python 2.7.10-final-0 -----------------------------------
Name                               Stmts   Miss  Cover
------------------------------------------------------
tesdat/__init__.py                    45      8    82%
tesdat/api/__init__.py                 0      0   100%
tesdat/api/display.py                 15      1    93%
tesdat/api/render.py                  23      4    83%
tesdat/api/special.py                  7      1    86%
tesdat/compat.py                      37     18    51%
tesdat/containers/__init__.py          0      0   100%
tesdat/containers/base.py             37      3    92%
tesdat/containers/dict.py             17      0   100%
tesdat/containers/iter.py             31      0   100%
tesdat/containers/list.py             19      1    95%
tesdat/containers/ordereddict.py       6      0   100%
tesdat/exceptions.py                   8      0   100%
tesdat/formatters/__init__.py          0      0   100%
tesdat/formatters/base.py             17      1    94%
tesdat/formatters/csv.py              42      5    88%
tesdat/formatters/json.py             20      0   100%
tesdat/formatters/pickle.py            9      0   100%
tesdat/formatters/string.py            7      0   100%
tesdat/models/__init__.py              0      0   100%
tesdat/models/base.py                  2      0   100%
tesdat/models/dict.py                 27      1    96%
tesdat/models/list.py                 34      1    97%
tesdat/patterns/__init__.py            0      0   100%
tesdat/patterns/base.py                3      0   100%
tesdat/patterns/choice.py             18      1    94%
tesdat/patterns/cycle.py              26      1    96%
tesdat/patterns/hashof.py             20      3    85%
tesdat/patterns/increment.py          16      2    88%
tesdat/patterns/pickout.py            25      2    92%
tesdat/patterns/sequence.py           30      1    97%
tesdat/utils/__init__.py               0      0   100%
tesdat/utils/datetime.py              48      6    88%
------------------------------------------------------
TOTAL                                589     60    90%

======================================== 57 passed, 1 skipped in 0.82 seconds =========================================
py27 runtests: commands[1] | coverage html
py33 recreate: /home/vagrant/tesdat/.tox/py33
py33 installdeps: pytest-cov
py33 inst: /home/vagrant/tesdat/.tox/dist/tesdat-2.1.1.zip
py33 installed: coverage==4.0.3,py==1.4.31,pytest==2.9.1,pytest-cov==2.2.1,tesdat==2.1.1
py33 runte
sts: PYTHONHASHSEED='2232003898'
py33 runtests: commands[0] | py.test --cov
================================================= test session starts =================================================
platform linux -- Python 3.3.6, pytest-2.9.1, py-1.4.31, pluggy-0.3.1
rootdir: /home/vagrant/tesdat, inifile:
plugins: cov-2.2.1
collected 58 items

tests-require.txt s
tesdat/tests/test_api.py ....
tesdat/tests/test_example.py ...
tesdat/tests/test_index.py ..
tesdat/tests/containers/test_base.py ...
tesdat/tests/containers/test_dict.py ...
tesdat/tests/containers/test_iter.py ...
tesdat/tests/containers/test_list.py ..
tesdat/tests/containers/test_ordereddict.py .
tesdat/tests/formatters/test_csv.py ...
tesdat/tests/formatters/test_json.py .
tesdat/tests/formatters/test_pickle.py .
tesdat/tests/formatters/test_string.py ..
tesdat/tests/models/test_dict.py .....
tesdat/tests/models/test_list.py ......
tesdat/tests/patterns/test_choice.py ..
tesdat/tests/patterns/test_cycle.py ..
tesdat/tests/patterns/test_hashof.py .
tesdat/tests/patterns/test_increment.py ..
tesdat/tests/patterns/test_pickout.py ..
tesdat/tests/patterns/test_sequence.py ..
tesdat/tests/utils/test_datetime.py .......
----------------------------------- coverage: platform linux, python 3.3.6-final-0 ------------------------------------
Name                               Stmts   Miss  Cover
------------------------------------------------------
tesdat/__init__.py                    45      8    82%
tesdat/api/__init__.py                 0      0   100%
tesdat/api/display.py                 15      1    93%
tesdat/api/render.py                  23      4    83%
tesdat/api/special.py                  7      1    86%
tesdat/compat.py                      37      4    89%
tesdat/containers/__init__.py          0      0   100%
tesdat/containers/base.py             37      3    92%
tesdat/containers/dict.py             17      0   100%
tesdat/containers/iter.py             31      0   100%
tesdat/containers/list.py             19      1    95%
tesdat/containers/ordereddict.py       6      0   100%
tesdat/exceptions.py                   8      0   100%
tesdat/formatters/__init__.py          0      0   100%
tesdat/formatters/base.py             17      1    94%
tesdat/formatters/csv.py              42      5    88%
tesdat/formatters/json.py             20      0   100%
tesdat/formatters/pickle.py            9      0   100%
tesdat/formatters/string.py            7      0   100%
tesdat/models/__init__.py              0      0   100%
tesdat/models/base.py                  2      0   100%
tesdat/models/dict.py                 27      1    96%
tesdat/models/list.py                 34      1    97%
tesdat/patterns/__init__.py            0      0   100%
tesdat/patterns/base.py                3      0   100%
tesdat/patterns/choice.py             18      1    94%
tesdat/patterns/cycle.py              26      1    96%
tesdat/patterns/hashof.py             20      3    85%
tesdat/patterns/increment.py          16      2    88%
tesdat/patterns/pickout.py            25      2    92%
tesdat/patterns/sequence.py           30      1    97%
tesdat/utils/__init__.py               0      0   100%
tesdat/utils/datetime.py              48      6    88%
------------------------------------------------------
TOTAL                                589     46    92%

======================================== 57 passed, 1 skipped in 1.09 seconds =========================================
py33 runtests: commands[1] | coverage html
py34 recreate: /home/vagrant/tesdat/.tox/py34
py34 installdeps: pytest-cov
py34 inst: /home/vagrant/tesdat/.tox/dist/tesdat-2.1.1.zip
py34 installed: coverage==4.0.3,py==1.4.31,pytest==2.9.1,pytest-cov==2.2.1,tesdat==2.1.1
py34 runtests: PYTHONHASHSEED='2232003898'
py34 runtests: commands[0] | py.test --cov
================================================= test session starts =================================================
platform linux -- Python 3.4.4, pytest-2.9.1, py-1.4.31, pluggy-0.3.1
rootdir: /home/vagrant/tesdat, inifile:
plugins: cov-2.2.1
collected 58 items

tests-require.txt s
tesdat/tests/test_api.py ....
tesdat/tests/test_example.py ...
tesdat/tests/test_index.py ..
tesdat/tests/containers/test_base.py ...
tesdat/tests/containers/test_dict.py ...
tesdat/tests/containers/test_iter.py ...
tesdat/tests/containers/test_list.py ..
tesdat/tests/containers/test_ordereddict.py .
tesdat/tests/formatters/test_csv.py ...
tesdat/tests/formatters/test_json.py .
tesdat/tests/formatters/test_pickle.py .
tesdat/tests/formatters/test_string.py ..
tesdat/tests/models/test_dict.py .....
tesdat/tests/models/test_list.py ......
tesdat/tests/patterns/test_choice.py ..
tesdat/tests/patterns/test_cycle.py ..
tesdat/tests/patterns/test_hashof.py .
tesdat/tests/patterns/test_increment.py ..
tesdat/tests/patterns/test_pickout.py ..
tesdat/tests/patterns/test_sequence.py ..
tesdat/tests/utils/test_datetime.py .......
----------------------------------- coverage: platform linux, python 3.4.4-final-0 ------------------------------------
Name                               Stmts   Miss  Cover
------------------------------------------------------
tesdat/__init__.py                    45      8    82%
tesdat/api/__init__.py                 0      0   100%
tesdat/api/display.py                 15      1    93%
tesdat/api/render.py                  23      4    83%
tesdat/api/special.py                  7      1    86%
tesdat/compat.py                      37      4    89%
tesdat/containers/__init__.py          0      0   100%
tesdat/containers/base.py             37      3    92%
tesdat/containers/dict.py             17      0   100%
tesdat/containers/iter.py             31      0   100%
tesdat/containers/list.py             19      1    95%
tesdat/containers/ordereddict.py       6      0   100%
tesdat/exceptions.py                   8      0   100%
tesdat/formatters/__init__.py          0      0   100%
tesdat/formatters/base.py             17      1    94%
tesdat/formatters/csv.py              42      5    88%
tesdat/formatters/json.py             20      0   100%
tesdat/formatters/pickle.py            9      0   100%
tesdat/formatters/string.py            7      0   100%
tesdat/models/__init__.py              0      0   100%
tesdat/models/base.py                  2      0   100%
tesdat/models/dict.py                 27      1    96%
tesdat/models/list.py                 34      1    97%
tesdat/patterns/__init__.py            0      0   100%
tesdat/patterns/base.py                3      0   100%
tesdat/patterns/choice.py             18      1    94%
tesdat/patterns/cycle.py              26      1    96%
tesdat/patterns/hashof.py             20      3    85%
tesdat/patterns/increment.py          16      2    88%
tesdat/patterns/pickout.py            25      2    92%
tesdat/patterns/sequence.py           30      1    97%
tesdat/utils/__init__.py               0      0   100%
tesdat/utils/datetime.py              48      6    88%
------------------------------------------------------
TOTAL                                589     46    92%

======================================== 57 passed, 1 skipped in 1.14 seconds =========================================
py34 runtests: commands[1] | coverage html
py35 recreate: /home/vagrant/tesdat/.tox/py35
py35 installdeps: pytest-cov
py35 inst: /home/vagrant/tesdat/.tox/dist/tesdat-2.1.1.zip
py35 installed: coverage==4.0.3,py==1.4.31,pytest==2.9.1,pytest-cov==2.2.1,tesdat==2.1.1
py35 runtests: PYTHONHASHSEED='2232003898'
py35 runtests: commands[0] | py.test --cov
================================================= test session starts =================================================
platform linux -- Python 3.5.1, pytest-2.9.1, py-1.4.31, pluggy-0.3.1
rootdir: /home/vagrant/tesdat, inifile:
plugins: cov-2.2.1
collected 58 items

tests-require.txt s
tesdat/tests/test_api.py ....
tesdat/tests/test_example.py ...
tesdat/tests/test_index.py ..
tesdat/tests/containers/test_base.py ...
tesdat/tests/containers/test_dict.py ...
tesdat/tests/containers/test_iter.py ...
tesdat/tests/containers/test_list.py ..
tesdat/tests/containers/test_ordereddict.py .
tesdat/tests/formatters/test_csv.py ...
tesdat/tests/formatters/test_json.py .
tesdat/tests/formatters/test_pickle.py .
tesdat/tests/formatters/test_string.py ..
tesdat/tests/models/test_dict.py .....
tesdat/tests/models/test_list.py ......
tesdat/tests/patterns/test_choice.py ..
tesdat/tests/patterns/test_cycle.py ..
tesdat/tests/patterns/test_hashof.py .
tesdat/tests/patterns/test_increment.py ..
tesdat/tests/patterns/test_pickout.py ..
tesdat/tests/patterns/test_sequence.py ..
tesdat/tests/utils/test_datetime.py .......
----------------------------------- coverage: platform linux, python 3.5.1-final-0 ------------------------------------
Name                               Stmts   Miss  Cover
------------------------------------------------------
tesdat/__init__.py                    45      8    82%
tesdat/api/__init__.py                 0      0   100%
tesdat/api/display.py                 15      1    93%
tesdat/api/render.py                  23      4    83%
tesdat/api/special.py                  7      1    86%
tesdat/compat.py                      37      4    89%
tesdat/containers/__init__.py          0      0   100%
tesdat/containers/base.py             37      3    92%
tesdat/containers/dict.py             17      0   100%
tesdat/containers/iter.py             31      0   100%
tesdat/containers/list.py             19      1    95%
tesdat/containers/ordereddict.py       6      0   100%
tesdat/exceptions.py                   8      0   100%
tesdat/formatters/__init__.py          0      0   100%
tesdat/formatters/base.py             17      1    94%
tesdat/formatters/csv.py              42      5    88%
tesdat/formatters/json.py             20      0   100%
tesdat/formatters/pickle.py            9      0   100%
tesdat/formatters/string.py            7      0   100%
tesdat/models/__init__.py              0      0   100%
tesdat/models/base.py                  2      0   100%
tesdat/models/dict.py                 27      1    96%
tesdat/models/list.py                 34      1    97%
tesdat/patterns/__init__.py            0      0   100%
tesdat/patterns/base.py                3      0   100%
tesdat/patterns/choice.py             18      1    94%
tesdat/patterns/cycle.py              26      1    96%
tesdat/patterns/hashof.py             20      3    85%
tesdat/patterns/increment.py          16      2    88%
tesdat/patterns/pickout.py            25      2    92%
tesdat/patterns/sequence.py           30      1    97%
tesdat/utils/__init__.py               0      0   100%
tesdat/utils/datetime.py              48      6    88%
------------------------------------------------------
TOTAL                                589     46    92%

======================================== 57 passed, 1 skipped in 1.08 seconds =========================================
py35 runtests: commands[1] | coverage html
flake8 recreate: /home/vagrant/tesdat/.tox/flake8
flake8 installdeps: flake8, flake8-blind-except, flake8-docstrings, flake8-import-order, mccabe, radon
flake8 inst: /home/vagrant/tesdat/.tox/dist/tesdat-2.1.1.zip
flake8 installed: colorama==0.3.7,flake8==2.5.4,flake8-blind-except==0.1.0,flake8-docstrings==0.2.6,flake8-import-order==0.7,mando==0.3.3,mccabe==0.4.0,pep257==0.7.0,pep8==1.7.0,pyflakes==1.0.0,radon==1.3.2,tesdat==2.1.1
flake8 runtests: PYTHONHASHSEED='2232003898'
flake8 runtests: commands[0] | flake8 .
_______________________________________________________ summary _______________________________________________________
  py27: commands succeeded
  py33: commands succeeded
  py34: commands succeeded
  py35: commands succeeded
  flake8: commands succeeded
  congratulations :)

htmlcov/index.html にアクセスすると以下のように表示されます。

埋め込める変数

上では説明しませんでしたが、 tox.ini 中には {spam} のように変数を埋め込むことができます。

重要そうな２つの変数については軽く動作を見ておきましょう。

posargs

posargsはtoxから渡された引数を参照します。例えばtox.iniのコマンドを以下のように書き換え

[testenv]
commands =
  py.test --cov . {posargs}

tox -e py35 tesdat/tests/test_api.py のように実行するとテストコマンドとしては py.test --cov . tesdat/tests/test_api.py のように展開されます。

pytestコマンドのテスト対象については pytestの記事で解説しています。

-- から始まる引数はtoxのオプションとして認識され、そんなのないよってことでエラーになっちゃうので注意してください。

(venv3.5) [vagrant@localhost tesdat]$ tox --aa
usage: tox [--version] [-h] [--help-ini] [-v] [--showconfig] [-l]
           [-c CONFIGFILE] [-e envlist] [--notest] [--sdistonly]
           [--installpkg PATH] [--develop] [-i URL] [--pre] [-r]
           [--result-json PATH] [--hashseed SEED] [--force-dep REQ]
           [--sitepackages] [--skip-missing-interpreters]
           [args [args ...]]
tox: error: unrecognized arguments: --aa

また、 pytest に対してオプションを渡したい場合は -- のあとに更にオプションを指定してください。

$ tox -- -k test_target

toxinidir

この変数はtox.iniがおいてあるディレクトリのパスを示します。

どこかに移動してからテストを実行したいとかであれば以下のように書けますね。

[testenv]
changedir = {toxinidir}/apps/
commands =
  py.test --cov .

toxについて記述がある数少ない書籍です。

<a href="https://www.amazon.co.jp/gp/product/4798053821/ref=as_li_ss_il?ie=UTF8&linkCode=li2&tag=7931-22&linkId=60383c998caacc23ad7bf98dbe028be2&language=ja_JP" target="_blank">
<img border="0" src="//ws-fe.amazon-adsystem.com/widgets/q?_encoding=UTF8&ASIN=4798053821&Format=_SL160_&ID=AsinImage&MarketPlace=JP&ServiceVersion=20070822&WS=1&tag=7931-22&language=ja_JP" ></a>
<img src="https://ir-jp.amazon-adsystem.com/e/ir?t=7931-22&language=ja_JP&l=li2&o=9&a=4798053821" width="1" height="1" border="0" alt="" style="border:none !important; margin:0px !important;" />