Skip to content
This repository was archived by the owner on Jul 22, 2019. It is now read-only.

Merge query server from kxepal/viewserver #276

Open
wants to merge 20 commits into
base: master
Choose a base branch
from

Conversation

iblislin
Copy link

Hi!
I read #268 and i really want the python query server!

This PR import the code from kxepal/couchdb-python@9145545.
And i modified the code in order to support python3.

The test cases from @kxepal is included.

To be honest, i do not test it with my couch yet. i just want to send this PR as soon as possible.
And i do not review whole the code base, i just make the testing pass.
So please test and review it carefully. 😄

I listed some idea needed everyone's feedback or potential problem here:

  • Test with pypy
    • performance benchmark
  • Threading/multiprocessing support
  • Security aspect
    • prevent dangerous import e.g. import socket, os
  • Documentation

iblislin added 18 commits March 13, 2016 01:32
- branch `viewserver`
- without any modification
checked with
$ pep8 --max-line-length=100 --show-source --first couchdb/server
…5545

- branch `viewserver`
- without any modification
checked with
$ pep8 --max-line-length=100 --show-source --first couchdb/server
Also, new file: tests/server/__main__ for standlone testing
In py3, the `base64.b64decode` will raise `binascii.Error`.
In py2, it will be `TypeError`
related test case:
server.mime.ProvidesTestCase.test_run_first_registered_for_unknown_mimetype
The origin test code compare ouput string directly,
but the order of python dict is random.

e.g. server.output == '{"foo": "bar", "baz": "qaz"}'
     If there are multiple keys, the comparison will fail randomly.

If there is only one key in result, i do not change the code.
The original `apply_context` will change `func.__globals__` which refers
to module globals().

Consider following senario:

- User has a script contains two functions: `mylist` and `myshow`.
- Also, the `mylist` can invoke `get_row`.
- There do not exist `get_row` in the scope of `myshow`, if user try to
  invoke it, the exception should raise.

Let's see what will original code do::

    def mylist():
        pass

    def myshow():
        get_row()  # this should fail

    render.run_list(..., mylist, ...)
    render.run_show(..., myshow, ...)
    # but then no exception raised

Because the first `run_list` will change `mylist.__globals__`,
and actually the `mylist.__globals__` and `myshow.__globals__` refer to
some object.
This pollutes the `__globals__`.
Also, fix the test cases in couchdb/tests/view.py
- We can run query server via
  `couchpy` or `python -m couchdb.server`
@kxepal
Copy link
Collaborator

kxepal commented Mar 15, 2016

Awesome! Thank you!

As for security: forget about it. There is no safe sandboxing for Python, unless PyPy, but it's not proven to be secure AFAIK.

@iblislin
Copy link
Author

Hi @kxepal
Long time age I have tried this https://pypi.python.org/pypi/RestrictedPython

Not sure this package is python3 ready or not, and it seems unmaintained.
Maybe we can try it out and fork it.

@kxepal
Copy link
Collaborator

kxepal commented Mar 15, 2016

Hi @iblis17
If you're interested there were quite comprehensive discussion about sanboxing Python on python-dev@ ML. Check "Reviving restricted mode?" and "The pysandbox project is broken" threads there (sorry, I'm not sure how to provide link to them with a nicer readable UI as my mail client). TL;DR - don't waste a time. Worth to check PyPy sandbox project though.

@iblislin
Copy link
Author

:-o I will check out PyPy sandbox project.

from pkgutil import iter_modules
except ImportError:
try:
# Python 2.4
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh, that was a time...I think this could be removed with light heart (:

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

removed in a0962ac

@djc
Copy link
Owner

djc commented Mar 15, 2016

Hey! I think it's great that you want to work on this. However, my concerns from #268 are pretty much still the same. Would you be up for creating a commit series following the plan outlined by @kxepal in that PR to bring us from the current state to your end state in a clean series?

@iblislin
Copy link
Author

Hi @djc!
Ok. I will make it. I need more time to understand the code for creating the commit series.

I will create a new branch to work on and let this branch un-rebased. Everyone can test/file bugs with this branch first.

@kxepal when I need your help, may I discuss here? or do we have irc channel?

@kxepal
Copy link
Collaborator

kxepal commented Mar 16, 2016

@iblis17 Feel free here anytime. Here we have a code as context, so it would be easy to talk about it.

iblislin pushed a commit to iblislin/couchdb-python that referenced this pull request May 15, 2016
Author:     Alexander Shorin <[email protected]>
Patched by: Iblis Lin <[email protected]>

Reference: djc#268
See Also:  djc#276
iblislin pushed a commit to iblislin/couchdb-python that referenced this pull request May 15, 2016
Author:     Alexander Shorin <[email protected]>
Patched by: Iblis Lin <[email protected]>

Reference: djc#268
See Also:  djc#276
iblislin pushed a commit to iblislin/couchdb-python that referenced this pull request May 15, 2016
Author:     Alexander Shorin <[email protected]>
Patched by: Iblis Lin <[email protected]>

Reference: djc#268
See Also:  djc#276
iblislin pushed a commit to iblislin/couchdb-python that referenced this pull request May 15, 2016
- Test case inclueded

Author:     Alexander Shorin <[email protected]>
Patched by: Iblis Lin <[email protected]>

Reference: djc#268
See Also:  djc#276
iblislin pushed a commit to iblislin/couchdb-python that referenced this pull request May 15, 2016
- Test cases included

Author:     Alexander Shorin <[email protected]>
Patched by: Iblis Lin <[email protected]>

Reference: djc#268
See Also:  djc#276
iblislin pushed a commit to iblislin/couchdb-python that referenced this pull request May 15, 2016
Author:     Alexander Shorin <[email protected]>
Patched by: Iblis Lin <[email protected]>

Reference: djc#268
See Also:  djc#276
iblislin added a commit to iblislin/couchdb-python that referenced this pull request May 20, 2016
The `output` parameter of `stream.respond` should be writable
with unicode.

Reference: djc#268
See Also:  djc#276
iblislin added a commit to iblislin/couchdb-python that referenced this pull request May 20, 2016
iblislin added a commit to iblislin/couchdb-python that referenced this pull request May 20, 2016
iblislin added a commit to iblislin/couchdb-python that referenced this pull request May 20, 2016
iblislin added a commit to iblislin/couchdb-python that referenced this pull request May 20, 2016
iblislin added a commit to iblislin/couchdb-python that referenced this pull request May 20, 2016
iblislin added a commit to iblislin/couchdb-python that referenced this pull request May 20, 2016
iblislin added a commit to iblislin/couchdb-python that referenced this pull request May 20, 2016
Drop the support for python25 or below

Reference: djc#268
See Also:  djc#276
iblislin added a commit to iblislin/couchdb-python that referenced this pull request May 21, 2016
We also take care the new log format.

Reference: djc#268
See Also:  djc#276
iblislin added a commit to iblislin/couchdb-python that referenced this pull request May 22, 2016
iblislin added a commit to iblislin/couchdb-python that referenced this pull request May 23, 2016
iblislin added a commit to iblislin/couchdb-python that referenced this pull request May 23, 2016
iblislin added a commit to iblislin/couchdb-python that referenced this pull request May 24, 2016
iblislin added a commit to iblislin/couchdb-python that referenced this pull request May 28, 2016
We take care the case of
"user really want a empty dictionary as `context`".

Reference: djc#268
See Also:  djc#276
iblislin added a commit to iblislin/couchdb-python that referenced this pull request May 28, 2016
iblislin added a commit to iblislin/couchdb-python that referenced this pull request May 30, 2016
compiler.
test_required_modules_has_global_namespace_access

Reference: djc#268
See Also:  djc#276
iblislin added a commit to iblislin/couchdb-python that referenced this pull request Jun 6, 2016
iblislin added a commit to iblislin/couchdb-python that referenced this pull request Jun 11, 2016
- The output is key-value pair only
- Also, provide comprehensive message

Reference: djc#268
See Also:  djc#276
iblislin added a commit to iblislin/couchdb-python that referenced this pull request Jun 23, 2016
iblislin pushed a commit to iblislin/couchdb-python that referenced this pull request Oct 25, 2016
Author:     Alexander Shorin <[email protected]>
Patched by: Iblis Lin <[email protected]>

Reference: djc#268
See Also:  djc#276
iblislin added a commit to iblislin/couchdb-python that referenced this pull request Oct 25, 2016
iblislin added a commit to iblislin/couchdb-python that referenced this pull request Oct 25, 2016
iblislin added a commit to iblislin/couchdb-python that referenced this pull request Oct 25, 2016
iblislin added a commit to iblislin/couchdb-python that referenced this pull request Oct 26, 2016
- Remove duplicate xhtml
- Introduce type aliases for `DEFAULT_TYPES`,
  e.g. `('text', 'txt'): ['text/plain; charset=utf-8']`.
  `txt` is an alias of `text`.

Reference: djc#268
See Also:  djc#276
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants