.
diff --git a/README.md b/README.md
index 25143744..50297bd8 100644
--- a/README.md
+++ b/README.md
@@ -1,117 +1,159 @@
-
-The Offensive Web Application Penetration Testing Framework.
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
+
+
+
+> __IMPORTANT__:
+>
+> The new Qt5 interface is complete, but has additional dependencies. Take a look at the updated installation instructions.
+
### Highlights :-
Here is some light on what the framework is all about:
-- [x] TIDoS Framework now boasts of a century+ of modules.
-- [x] A complete versatile framework to cover up everything from Reconnaissance to Vulnerability Analysis.
-- [x] Has 5 main phases, subdivided into __14 sub-phases__ consisting a total of __108 modules__ .
-- [x] Reconnaissance Phase has 50 modules of its own (including active and passive recon, information disclosure modules).
-- [x] Scanning & Enumeration Phase has got 16 modules (including port scans, WAF analysis, etc)
-- [x] Vulnerability Analysis Phase has 37 modules (including most common vulnerabilites in action).
-- [x] Exploits Castle has only 1 exploit. `(purely developmental)`
-- [x] And finally, Auxillaries have got 4 modules. `more under development`
-- [x] All four phases each have a `Auto-Awesome` module which automates every module for you.
-- [x] You just need the domain, and leave everything is to this tool.
-- [x] TIDoS has full verbose out support, so you'll know whats going on.
-- [x] Fully user friendly interaction environment. `(no shits)`
-
-\fR" or "\fB? \fR". To list all available commands, use "\fBhelp\fR" resp. "\fB?\fR" without argument. This will also give a short summary on the commands, as listed below.
+
+.P
+\fICommands:\fR
+
+.TP
+\fBattack\fR
+Attack specified target(s)
+.TP
+\fBclear\fR
+Clear terminal
+.TP
+\fBcreds\fR
+Handle target credentials
+.TP
+\fBfetch\fR
+Check for and install updates
+.TP
+\fBfind\fR
+Search a module
+.TP
+\fBhelp\fR
+Show help message
+.TP
+\fBinfo\fR
+Show description of current module
+.TP
+\fBintro\fR
+Display Intro
+.TP
+\fBleave\fR
+Leave module
+.TP
+\fBlist\fR
+List all modules of a category
+.TP
+\fBload\fR
+Load module
+.TP
+\fBnetinfo\fR
+Show network information
+.TP
+\fBopts\fR
+Show options of current module
+.TP
+\fBphpsploit\fR
+Load the phpsploit framework (needs to be downloaded externally)
+.TP
+\fBprocesses\fR
+Set number of processes in parallelisation
+.TP
+\fBq\fR
+Terminate TIDoS session
+.TP
+\fBsessions\fR
+Interact with cached sessions
+.TP
+\fBset\fR
+Set option value of module
+.TP
+\fBtor\fR
+Pipe Attacks through the Tor Network
+.TP
+\fBvicadd\fR
+Add Target to list
+.TP
+\fBvicdel\fR
+Delete Target from list
+.TP
+\fBviclist\fR
+List all targets
+
+.SH ISSUES
+.TP
+To submit any issue, bug or proposal, please send it in TIDoS's issues section:
+https://github.com/VainlyStrain/TIDoS/issues
+
+.SH CONTRIBUTE
+.TP
+If you want to contribute to \fBVaile\fR, submit a module, patch, or anything else, take a look at the \fBCONTRIBUTING\fR file, in the core/doc directory
+
+.SH LICENCE
+.P
+This software is under the GNU GENERAL PUBLIC LICENSE Version 3, 29 June 2007
+
+
+.\" man code generated by txt2tags 2.5 (http://txt2tags.sf.net)
+.\" cmdline: txt2tags -q -t man -i man.txt2tags -o TIDoS.1
+
diff --git a/core/doc/mystery b/core/doc/mystery
new file mode 100644
index 00000000..396a0ba2
--- /dev/null
+++ b/core/doc/mystery
@@ -0,0 +1 @@
+yes
\ No newline at end of file
diff --git a/core/doc/vailyn b/core/doc/vailyn
new file mode 100644
index 00000000..8d1c8b69
--- /dev/null
+++ b/core/doc/vailyn
@@ -0,0 +1 @@
+
diff --git a/core/doc/version b/core/doc/version
new file mode 100644
index 00000000..52837484
--- /dev/null
+++ b/core/doc/version
@@ -0,0 +1 @@
+2.0.5-1
diff --git a/core/docker/Dockerfile b/core/docker/Dockerfile
new file mode 100644
index 00000000..309b40fa
--- /dev/null
+++ b/core/docker/Dockerfile
@@ -0,0 +1,41 @@
+# Pulling base image
+FROM ubuntu:18.04
+
+# Install TIDoS-Framework dependencies
+RUN apt update && \
+ apt install --install-recommends -y \
+ sudo \
+ libncurses5 \
+ apt-utils \
+ dialog \
+ libxml2 \
+ nmap \
+ git \
+ nano \
+ xcb \
+ tcpdump \
+ libexiv2-dev \
+ build-essential \
+ python-xmpp \
+ python-dev \
+ python3-pip \
+ libmysqlclient-dev \
+ tor \
+ konsole \
+ && \
+ rm -rf /var/lib/apt/lists/* && \
+ apt-get clean
+
+# Installing TIDoS-Framework
+RUN git clone https://github.com/0xinfection/tidos-framework.git && \
+ cd tidos-framework && \
+ python3 -m pip install --upgrade --force pip && \
+ python3 -m pip install --upgrade --force wheel && \
+ python3 -m pip install -r requirements.txt && \
+ mkdir -v -p /opt/TIDoS/ && \
+ cp -r -v * /opt/TIDoS/ && \
+ cp -v tmp/tidos /usr/bin/tidos && \
+ export EDITOR=nano && \
+ chmod -R 755 /opt/TIDoS/* && \
+ chmod -v 755 /usr/bin/tidos && \
+ cd .. && rm -rf tidos-framework
diff --git a/core/initfiles/sample.vaile b/core/initfiles/sample.vaile
new file mode 100644
index 00000000..21098bb8
--- /dev/null
+++ b/core/initfiles/sample.vaile
@@ -0,0 +1,3 @@
+sessions load --val saarsec.val
+load pathtrav
+attack
diff --git a/core/install.py b/core/install.py
new file mode 100755
index 00000000..5b40110c
--- /dev/null
+++ b/core/install.py
@@ -0,0 +1,45 @@
+#!/usr/bin/env python3
+# -*- coding: utf-8 -*-
+"""
+_____, ___
+ '+ .;
+ , ;
+ .
+
+ .
+ .;.
+ .;
+ :
+ ,
+
+
+┌─[TIDoS]─[]
+└──╼ VainlyStrain
+"""
+
+import os
+
+print('''\033[1m
+ ___ _ _ _
+ |_ _| _ _ ___ | |_ __ _ | | | |
+ | | | ' \\ (_-< | _| / _` | | | | |
+ |___| |_||_| /__/_ _\\__| \\__,_| _|_|_ _|_|_
+_|"""""|_|"""""|_|"""""|_|"""""|_|"""""|_|"""""|_|"""""|
+"`-0-0-'"`-0-0-'"`-0-0-'"`-0-0-'"`-0-0-'"`-0-0-'"`-0-0-'
+''')
+
+
+print(" [+] Installing dependencies (1/2): Package Manager")
+os.system("apt-get install libncurses5 libxml2 nmap tcpdump libexiv2-dev build-essential python3-pip libmariadbclient18 libmysqlclient-dev tor konsole")
+
+print(" [+] Installing dependencies (2/2): pip3")
+os.system("pip3 install -r requirements.txt")
+
+print(" [+] Installing TIDoS...")
+os.system('mkdir -v -p /opt/TIDoS/')
+os.system('cp -r -v ../* /opt/TIDoS/')
+os.system('cp -v ../tmp/tidos /usr/bin/tidos')
+os.system('chmod -R 755 /opt/TIDoS/*')
+os.system('chmod -v 755 /usr/bin/tidos')
+
+print("Installation process complete. Run 'tidos' to launch the framework.\033[0m")
diff --git a/core/lib/FileUtils.py b/core/lib/FileUtils.py
new file mode 100644
index 00000000..aa3ed0c0
--- /dev/null
+++ b/core/lib/FileUtils.py
@@ -0,0 +1,138 @@
+#!/usr/bin/env python
+# -*- coding: utf-8 -*-
+
+#-:-:-:-:-:-:-:-:-:-:-:-:#
+# TIDoS Framework #
+#-:-:-:-:-:-:-:-:-:-:-:-:#
+
+#This module requires TIDoS Framework
+#https://github.com/0xInfection/TIDoS-Framework
+
+import os
+import os.path
+
+
+class File(object):
+ def __init__(self, *pathComponents):
+ self._path = FileUtils.buildPath(*pathComponents)
+ self.content = None
+
+ @property
+ def path(self):
+ return self._path
+
+ @path.setter
+ def path(self, value):
+ raise NotImplemented
+
+ def isValid(self):
+ return FileUtils.isFile(self.path)
+
+ def exists(self):
+ return FileUtils.exists(self.path)
+
+ def canRead(self):
+ return FileUtils.canRead(self.path)
+
+ def canWrite(self):
+ return FileUtils.canWrite(self.path)
+
+ def read(self):
+ return FileUtils.read(self.path)
+
+ def update(self):
+ self.content = self.read()
+
+ def content(self):
+ if not self.content:
+ self.content = FileUtils.read()
+ return self.content()
+
+ def getLines(self):
+ for line in FileUtils.getLines(self.path):
+ yield line
+
+ def __cmp__(self, other):
+ if not isinstance(other, File):
+ raise NotImplemented
+ return cmp(self.content(), other.content())
+
+ def __enter__(self):
+ return self
+
+ def __exit__(self, type, value, tb):
+ pass
+
+
+class FileUtils(object):
+ @staticmethod
+ def buildPath(*pathComponents):
+ if pathComponents:
+ path = os.path.join(*pathComponents)
+ else:
+ path = ''
+ return path
+
+ @staticmethod
+ def exists(fileName):
+ return os.access(fileName, os.F_OK)
+
+ @staticmethod
+ def canRead(fileName):
+ if not os.access(fileName, os.R_OK):
+ return False
+ try:
+ with open(fileName):
+ pass
+ except IOError:
+ return False
+ return True
+
+ @staticmethod
+ def canWrite(fileName):
+ return os.access(fileName, os.W_OK)
+
+ @staticmethod
+ def read(fileName):
+ result = ''
+ with open(fileName, 'r') as fd:
+ for line in fd.readlines():
+ result += line
+ return result
+
+ @staticmethod
+ def getLines(fileName):
+ with open(fileName, 'r', errors="replace") as fd:
+ return fd.read().splitlines()
+
+ @staticmethod
+ def isDir(fileName):
+ return os.path.isdir(fileName)
+
+ @staticmethod
+ def isFile(fileName):
+ return os.path.isfile(fileName)
+
+ @staticmethod
+ def createDirectory(directory):
+ if not FileUtils.exists(directory):
+ os.makedirs(directory)
+
+ @staticmethod
+ def sizeHuman(num):
+ base = 1024
+ for x in ['B ', 'KB', 'MB', 'GB']:
+ if num < base and num > -base:
+ return "%3.0f%s" % (num, x)
+ num /= base
+ return "%3.0f %s" % (num, 'TB')
+
+ @staticmethod
+ def writeLines(fileName, lines):
+ content = None
+ if type(lines) is list:
+ content = "\n".join(lines)
+ else:
+ content = lines
+ with open(fileName, "w") as f:
+ f.writelines(content)
diff --git a/core/lib/Photon/.travis.yml b/core/lib/Photon/.travis.yml
new file mode 100644
index 00000000..b5731398
--- /dev/null
+++ b/core/lib/Photon/.travis.yml
@@ -0,0 +1,16 @@
+language: python
+os:
+ - linux
+python:
+ - 3.6
+install:
+ - pip install -r requirements.txt
+ - pip install flake8
+before_script:
+ # stop the build if there are Python syntax errors or undefined names
+ - flake8 . --count --select=E901,E999,F821,F822,F823 --show-source --statistics
+ # exit-zero treats all errors as warnings. The GitHub editor is 127 chars wide
+ - flake8 . --count --exit-zero --max-complexity=10 --max-line-length=127 --statistics
+script:
+ - python photon.py -u "https://somdev.me/" -l 1 -d 1 -t 100 --regex "\d{10}" --dns --output="d3v"
+ - python photon.py -u "https://somdev.me/" -l 1 -t 10 --seeds="https://somdev.me/posts" --only-urls --export=json --wayback
diff --git a/core/lib/Photon/.whitesource b/core/lib/Photon/.whitesource
new file mode 100644
index 00000000..f0569521
--- /dev/null
+++ b/core/lib/Photon/.whitesource
@@ -0,0 +1,8 @@
+{
+ "generalSettings": {
+ "shouldScanRepo": true
+ },
+ "checkRunSettings": {
+ "vulnerableCheckRunConclusionLevel": "failure"
+ }
+}
\ No newline at end of file
diff --git a/core/lib/Photon/CHANGELOG.md b/core/lib/Photon/CHANGELOG.md
new file mode 100644
index 00000000..cc5fdad3
--- /dev/null
+++ b/core/lib/Photon/CHANGELOG.md
@@ -0,0 +1,121 @@
+#### v1.3.2
+- add support for socks proxies
+- add rotating proxies
+- `-p` now takes `IP:PORT` or `DOMAIN:PORT` with `http://`, `socks5://` or nothing (default is `http`) or a `file` with a list of proxies (`http://` or `socks5://` or nothing).
+
+#### v1.3.1
+- Added more intels (GENERIC_URL, BRACKET_URL, BACKSLASH_URL, HEXENCODED_URL, URLENCODED_URL, B64ENCODED_URL, IPV4, IPV6, EMAIL, MD5, SHA1, SHA256, SHA512, YARA_PARSE, CREDIT_CARD)
+- proxy support with `-p, --proxy` option (http proxy only)
+- minor fixes and pep8 format
+
+#### v1.3.0
+- Dropped Python < 3.2 support
+- Removed Ninja mode
+- Fixed a bug in link parsing
+- Fixed Unicode output
+- Fixed a bug which caused URLs to be treated as files
+- Intel is now associated with the URL where it was found
+
+#### v1.2.1
+- Added cloning ability
+- Refactored to be modular
+
+#### v1.1.6
+- Reuse TCP connection for better performance
+- Handle redirect loops
+- CSV export support
+- Fixed `sitemap.xml` parsing
+- Improved regex
+
+#### v1.1.5
+- fixed some minor bugs
+- fixed a bug in domain name parsing
+- added --headers option for interactive HTTP headers input
+
+#### v1.1.4
+- Added `-v` option
+- Fixed progress animation for Python 2
+- Added `developer.facebook.com` API for Ninja mode
+
+#### v1.1.3
+- Added `--stdout` option
+- Fixed a bug in `zap()` function
+- Fixed crashing when target is an IP address
+- Minor refactor
+
+#### v1.1.2
+- Added `--wayback`
+- Fixed progress bar for Python > 3.2
+- Added `/core/config.py` for easy customization
+- `--dns` now saves subdomains in `subdomains.txt`
+
+#### v1.1.1
+- Use of `ThreadPoolExecutor` for x2 speed (for Python > 3.2)
+- Fixed mishandling of urls starting with `//`
+- Removed a redundant try-except statement
+- Evaluate entropy of found keys to avoid false positives
+
+#### v1.1.0
+- Added `--keys` option
+- Fixed a bug related to SSL certificate verification
+
+#### v1.0.9
+- Code refactor
+- Better identification of external URLs
+- Fixed a major bug that made several intel URLs pass under the radar
+- Fixed a major bug that caused non-html type content to be marked a crawlable URL
+
+#### v1.0.8
+- added `--exclude` option
+- Better regex and code logic to favor performance
+- Fixed a bug that caused dnsdumpster to fail if target was a subdomain
+- Fixed a bug that caused a crash if run outside "Photon" directory
+- Fixed a bug in file saving (specific to Python 3)
+
+#### v1.0.7
+- Added `--timeout` option
+- Added `--output` option
+- Added `--user-agent` option
+- Replaced lxml with regex
+- Better logic for favoring performance
+- Added bigger and separate file for user-agents
+
+#### v1.0.6
+- Fixed lot of bugs
+- Suppress SSL warnings in MAC
+- x100 speed by code optimization
+- Simplified code of `exporter` plugin
+
+#### v1.0.5
+- Added `exporter` plugin
+- Added seamless update ability
+- Fixed a bug in update function
+
+#### v1.0.4
+- Fixed an issue which caused regular links to be saved in robots.txt
+- Simplified `flash` function
+- Removed `-n` as an alias of `--ninja`
+- Added `--only-urls` option
+- Refactored code for readability
+- Skip saving files if the content is empty
+
+#### v1.0.3
+- Introduced plugins
+- Added `dnsdumpster` plugin
+- Fixed non-ascii character handling, again
+- 404 pages are now added to `failed` list
+- Handling exceptions in `jscanner`
+
+#### v1.0.2
+- Proper handling of null response from `robots.txt` & `sitemap.xml`
+- Python2 compatibility
+- Proper handling of non-ascii chars
+- Added ability to specify custom regex pattern
+- Display total time taken and average time per request
+
+#### v1.0.1
+- Disabled colors on Windows and macOS
+- Cross platform file handling
+
+#### v1.0.0
+- First stable release
diff --git a/core/lib/Photon/Dockerfile b/core/lib/Photon/Dockerfile
new file mode 100644
index 00000000..4b9fc881
--- /dev/null
+++ b/core/lib/Photon/Dockerfile
@@ -0,0 +1,16 @@
+FROM python:3-alpine
+
+LABEL name photon
+LABEL src "https://github.com/s0md3v/Photon"
+LABEL creator s0md3v
+LABEL dockerfile_maintenance khast3x
+LABEL desc "Incredibly fast crawler designed for reconnaissance."
+
+RUN apk add git && git clone https://github.com/s0md3v/Photon.git Photon
+WORKDIR Photon
+RUN pip install -r requirements.txt
+
+VOLUME [ "/Photon" ]
+# ENTRYPOINT ["sh"]
+ENTRYPOINT [ "python", "photon.py" ]
+CMD ["--help"]
diff --git a/core/lib/Photon/LICENSE.md b/core/lib/Photon/LICENSE.md
new file mode 100644
index 00000000..deea74ad
--- /dev/null
+++ b/core/lib/Photon/LICENSE.md
@@ -0,0 +1,674 @@
+ GNU GENERAL PUBLIC LICENSE
+ Version 3, 29 June 2007
+
+ Copyright (C) 2007 Free Software Foundation, Inc.
+ Copyright (C)
+
+ This program is free software: you can redistribute it and/or modify
+ it under the terms of the GNU General Public License as published by
+ the Free Software Foundation, either version 3 of the License, or
+ (at your option) any later version.
+
+ This program is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ GNU General Public License for more details.
+
+ You should have received a copy of the GNU General Public License
+ along with this program. If not, see .
\ No newline at end of file
diff --git a/core/lib/Photon/MANIFEST.in b/core/lib/Photon/MANIFEST.in
new file mode 100644
index 00000000..e267daf0
--- /dev/null
+++ b/core/lib/Photon/MANIFEST.in
@@ -0,0 +1 @@
+include LICENSE.md
diff --git a/core/lib/Photon/README.md b/core/lib/Photon/README.md
new file mode 100644
index 00000000..bf1de88d
--- /dev/null
+++ b/core/lib/Photon/README.md
@@ -0,0 +1,108 @@
+
+
+
+
+Incredibly fast crawler designed for OSINT.
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ Photon Wiki •
+ How To Use •
+ Compatibility •
+ Photon Library •
+ Contribution •
+ Roadmap
+
+
+### Key Features
+
+#### Data Extraction
+Photon can extract the following data while crawling:
+
+- URLs (in-scope & out-of-scope)
+- URLs with parameters (`example.com/gallery.php?id=2`)
+- Intel (emails, social media accounts, amazon buckets etc.)
+- Files (pdf, png, xml etc.)
+- Secret keys (auth/API keys & hashes)
+- JavaScript files & Endpoints present in them
+- Strings matching custom regex pattern
+- Subdomains & DNS related data
+
+The extracted information is saved in an organized manner or can be [exported as json](https://github.com/s0md3v/Photon/wiki/Usage#export-formatted-result).
+
+
+
+#### Flexible
+Control timeout, delay, add seeds, exclude URLs matching a regex pattern and other cool stuff.
+The extensive range of [options](https://github.com/s0md3v/Photon/wiki/Usage) provided by Photon lets you crawl the web exactly the way you want.
+
+#### Genius
+Photon's smart thread management & refined logic gives you top notch performance.
+
+Still, crawling can be resource intensive but Photon has some tricks up it's sleeves. You can fetch URLs archived by [archive.org](https://archive.org/) to be used as seeds by using `--wayback` option.
+
+#### Plugins
+- **[wayback](https://github.com/s0md3v/Photon/wiki/Usage#use-urls-from-archiveorg-as-seeds)**
+- **[dnsdumpster](https://github.com/s0md3v/Photon/wiki/Usage#dumping-dns-data)**
+- **[Exporter](https://github.com/s0md3v/Photon/wiki/Usage#export-formatted-result)**
+
+#### Docker
+
+Photon can be launched using a lightweight Python-Alpine (103 MB) Docker image.
+
+```bash
+$ git clone https://github.com/s0md3v/Photon.git
+$ cd Photon
+$ docker build -t photon .
+$ docker run -it --name photon photon:latest -u google.com
+```
+
+To view results, you can either head over to the local docker volume, which you can find by running `docker inspect photon` or by mounting the target loot folder:
+
+```bash
+$ docker run -it --name photon -v "$PWD:/Photon/google.com" photon:latest -u google.com
+```
+
+#### Frequent & Seamless Updates
+Photon is under heavy development and updates for fixing bugs. optimizing performance & new features are being rolled regularly.
+
+If you would like to see features and issues that are being worked on, you can do that on [Development](https://github.com/s0md3v/Photon/projects/1) project board.
+
+Updates can be installed & checked for with the `--update` option. Photon has seamless update capabilities which means you can update Photon without losing any of your saved data.
+
+### Contribution & License
+You can contribute in following ways:
+
+- Report bugs
+- Develop plugins
+- Add more "APIs" for ninja mode
+- Give suggestions to make it better
+- Fix issues & submit a pull request
+
+Please read the [guidelines](https://github.com/s0md3v/Photon/wiki/Guidelines) before submitting a pull request or issue.
+
+Do you want to have a conversation in private? Hit me up on my [twitter](https://twitter.com/s0md3v/), inbox is open :)
+
+**Photon** is licensed under [GPL v3.0 license](https://www.gnu.org/licenses/gpl-3.0.en.html)
diff --git a/core/lib/Photon/core/__init__.py b/core/lib/Photon/core/__init__.py
new file mode 100644
index 00000000..0d75f5e1
--- /dev/null
+++ b/core/lib/Photon/core/__init__.py
@@ -0,0 +1 @@
+"""The Photon core."""
diff --git a/core/lib/Photon/core/colors.py b/core/lib/Photon/core/colors.py
new file mode 100644
index 00000000..f549f64b
--- /dev/null
+++ b/core/lib/Photon/core/colors.py
@@ -0,0 +1,17 @@
+import sys
+
+if sys.platform.lower().startswith(('os', 'win', 'darwin', 'ios')):
+ # Colors shouldn't be displayed on Mac and Windows
+ end = red = white = green = yellow = run = bad = good = info = que = ''
+else:
+ white = '\033[97m'
+ green = '\033[92m'
+ red = '\033[91m'
+ yellow = '\033[93m'
+ end = '\033[0m'
+ back = '\033[7;91m'
+ info = '\033[93m[!]\033[0m'
+ que = '\033[94m[?]\033[0m'
+ bad = '\033[91m[-]\033[0m'
+ good = '\033[92m[+]\033[0m'
+ run = '\033[97m[~]\033[0m'
diff --git a/core/lib/Photon/core/config.py b/core/lib/Photon/core/config.py
new file mode 100644
index 00000000..0a54b9ab
--- /dev/null
+++ b/core/lib/Photon/core/config.py
@@ -0,0 +1,27 @@
+"""Configuration options for Photon."""
+
+VERBOSE = False
+
+INTELS = [
+ 'facebook.com',
+ 'github.com',
+ 'instagram.com',
+ 'youtube.com',
+]
+
+BAD_TYPES = (
+ 'bmp',
+ 'css',
+ 'csv',
+ 'docx',
+ 'ico',
+ 'jpeg',
+ 'jpg',
+ 'js',
+ 'json',
+ 'pdf',
+ 'png',
+ 'svg',
+ 'xls',
+ 'xml',
+)
diff --git a/core/lib/Photon/core/flash.py b/core/lib/Photon/core/flash.py
new file mode 100644
index 00000000..0741fbba
--- /dev/null
+++ b/core/lib/Photon/core/flash.py
@@ -0,0 +1,20 @@
+# from __future__ import print_function
+import concurrent.futures
+import sys
+import os
+from os import path
+sys.path.append(os.path.abspath('.'))
+
+from .colors import info
+
+def flash(function, links, thread_count):
+ """Process the URLs and uses a threadpool to execute a function."""
+ # Convert links (set) to list
+ links = list(links)
+ threadpool = concurrent.futures.ThreadPoolExecutor(
+ max_workers=thread_count)
+ futures = (threadpool.submit(function, link) for link in links)
+ for i, _ in enumerate(concurrent.futures.as_completed(futures)):
+ if i + 1 == len(links) or (i + 1) % thread_count == 0:
+ print('%s Progress: %i/%i' % (info, i + 1, len(links)))
+ print('')
diff --git a/core/lib/Photon/core/mirror.py b/core/lib/Photon/core/mirror.py
new file mode 100644
index 00000000..8dfebe06
--- /dev/null
+++ b/core/lib/Photon/core/mirror.py
@@ -0,0 +1,39 @@
+import os
+
+
+def mirror(url, response):
+ if response != 'dummy':
+ clean_url = url.replace('http://', '').replace('https://', '').rstrip('/')
+ parts = clean_url.split('?')[0].split('/')
+ root = parts[0]
+ webpage = parts[-1]
+ parts.remove(root)
+ try:
+ parts.remove(webpage)
+ except ValueError:
+ pass
+ prefix = root + '_mirror'
+ try:
+ os.mkdir(prefix)
+ except OSError:
+ pass
+ suffix = ''
+ if parts:
+ for directory in parts:
+ suffix += directory + '/'
+ try:
+ os.mkdir(prefix + '/' + suffix)
+ except OSError:
+ pass
+ path = prefix + '/' + suffix
+ trail = ''
+ if '.' not in webpage:
+ trail += '.html'
+ if webpage == root:
+ name = 'index.html'
+ else:
+ name = webpage
+ if len(url.split('?')) > 1:
+ trail += '?' + url.split('?')[1]
+ with open(path + name + trail, 'w+') as out_file:
+ out_file.write(response.encode('utf-8'))
diff --git a/core/lib/Photon/core/prompt.py b/core/lib/Photon/core/prompt.py
new file mode 100644
index 00000000..0e3e84bf
--- /dev/null
+++ b/core/lib/Photon/core/prompt.py
@@ -0,0 +1,22 @@
+"""Support for an input prompt."""
+import os
+import tempfile
+
+
+def prompt(default=None):
+ """Present the user a prompt."""
+ editor = 'nano'
+ with tempfile.NamedTemporaryFile(mode='r+') as tmpfile:
+ if default:
+ tmpfile.write(default)
+ tmpfile.flush()
+
+ child_pid = os.fork()
+ is_child = child_pid == 0
+
+ if is_child:
+ os.execvp(editor, [editor, tmpfile.name])
+ else:
+ os.waitpid(child_pid, 0)
+ tmpfile.seek(0)
+ return tmpfile.read().strip()
diff --git a/core/lib/Photon/core/regex.py b/core/lib/Photon/core/regex.py
new file mode 100644
index 00000000..570742d8
--- /dev/null
+++ b/core/lib/Photon/core/regex.py
@@ -0,0 +1,234 @@
+import re
+
+# regex taken from https://github.com/InQuest/python-iocextract
+# Reusable end punctuation regex.
+END_PUNCTUATION = r"[\.\?>\"'\)!,}:;\u201d\u2019\uff1e\uff1c\]]*"
+
+# Reusable regex for symbols commonly used to defang.
+SEPARATOR_DEFANGS = r"[\(\)\[\]{}<>\\]"
+
+# Split URLs on some characters that may be valid, but may also be garbage.
+URL_SPLIT_STR = r"[>\"'\),};]"
+
+# Get basic url format, including a few obfuscation techniques, main anchor is the uri scheme.
+GENERIC_URL = re.compile(r"""
+ (
+ # Scheme.
+ [fhstu]\S\S?[px]s?
+ # One of these delimiters/defangs.
+ (?:
+ :\/\/|
+ :\\\\|
+ :?__
+ )
+ # Any number of defang characters.
+ (?:
+ \x20|
+ """ + SEPARATOR_DEFANGS + r"""
+ )*
+ # Domain/path characters.
+ \w
+ \S+?
+ # CISCO ESA style defangs followed by domain/path characters.
+ (?:\x20[\/\.][^\.\/\s]\S*?)*
+ )
+ """ + END_PUNCTUATION + r"""
+ (?=\s|$)
+ """, re.IGNORECASE | re.VERBOSE | re.UNICODE)
+
+# Get some obfuscated urls, main anchor is brackets around the period.
+BRACKET_URL = re.compile(r"""
+ \b
+ (
+ [\.\:\/\\\w\[\]\(\)-]+
+ (?:
+ \x20?
+ [\(\[]
+ \x20?
+ \.
+ \x20?
+ [\]\)]
+ \x20?
+ \S*?
+ )+
+ )
+ """ + END_PUNCTUATION + r"""
+ (?=\s|$)
+ """, re.VERBOSE | re.UNICODE)
+
+# Get some obfuscated urls, main anchor is backslash before a period.
+BACKSLASH_URL = re.compile(r"""
+ \b
+ (
+ [\:\/\\\w\[\]\(\)-]+
+ (?:
+ \x20?
+ \\?\.
+ \x20?
+ \S*?
+ )*?
+ (?:
+ \x20?
+ \\\.
+ \x20?
+ \S*?
+ )
+ (?:
+ \x20?
+ \\?\.
+ \x20?
+ \S*?
+ )*
+ )
+ """ + END_PUNCTUATION + r"""
+ (?=\s|$)
+ """, re.VERBOSE | re.UNICODE)
+
+# Get hex-encoded urls.
+HEXENCODED_URL = re.compile(r"""
+ (
+ [46][86]
+ (?:[57]4)?
+ [57]4[57]0
+ (?:[57]3)?
+ 3a2f2f
+ (?:2[356def]|3[0-9adf]|[46][0-9a-f]|[57][0-9af])+
+ )
+ (?:[046]0|2[0-2489a-c]|3[bce]|[57][b-e]|[8-f][0-9a-f]|0a|0d|09|[
+ \x5b-\x5d\x7b\x7d\x0a\x0d\x20
+ ]|$)
+ """, re.IGNORECASE | re.VERBOSE)
+
+# Get urlencoded urls.
+URLENCODED_URL = re.compile(r"""
+ (s?[hf]t?tps?%3A%2F%2F\w[\w%-]*?)(?:[^\w%-]|$)
+ """, re.IGNORECASE | re.VERBOSE)
+
+# Get base64-encoded urls.
+B64ENCODED_URL = re.compile(r"""
+ (
+ # b64re '([hH][tT][tT][pP][sS]|[hH][tT][tT][pP]|[fF][tT][pP])://'
+ # Modified to ignore whitespace.
+ (?:
+ [\x2b\x2f-\x39A-Za-z]\s*[\x2b\x2f-\x39A-Za-z]\s*[\x31\x35\x39BFJNRVZdhlptx]\s*[Gm]\s*[Vd]\s*[FH]\s*[A]\s*\x36\s*L\s*y\s*[\x2b\x2f\x38-\x39]\s*|
+ [\x2b\x2f-\x39A-Za-z]\s*[\x2b\x2f-\x39A-Za-z]\s*[\x31\x35\x39BFJNRVZdhlptx]\s*[Io]\s*[Vd]\s*[FH]\s*[R]\s*[Qw]\s*[O]\s*i\s*\x38\s*v\s*[\x2b\x2f-\x39A-Za-z]\s*|
+ [\x2b\x2f-\x39A-Za-z]\s*[\x2b\x2f-\x39A-Za-z]\s*[\x31\x35\x39BFJNRVZdhlptx]\s*[Io]\s*[Vd]\s*[FH]\s*[R]\s*[Qw]\s*[Uc]\s*[z]\s*o\s*v\s*L\s*[\x2b\x2f-\x39w-z]\s*|
+ [\x2b\x2f-\x39A-Za-z]\s*[\x30\x32EGUWkm]\s*[Z]\s*[\x30U]\s*[Uc]\s*[D]\s*o\s*v\s*L\s*[\x2b\x2f-\x39w-z]\s*|
+ [\x2b\x2f-\x39A-Za-z]\s*[\x30\x32EGUWkm]\s*[h]\s*[\x30U]\s*[Vd]\s*[FH]\s*[A]\s*\x36\s*L\s*y\s*[\x2b\x2f\x38-\x39]\s*|
+ [\x2b\x2f-\x39A-Za-z]\s*[\x30\x32EGUWkm]\s*[h]\s*[\x30U]\s*[Vd]\s*[FH]\s*[B]\s*[Tz]\s*[O]\s*i\s*\x38\s*v\s*[\x2b\x2f-\x39A-Za-z]\s*|
+ [RZ]\s*[ln]\s*[R]\s*[Qw]\s*[O]\s*i\s*\x38\s*v\s*[\x2b\x2f-\x39A-Za-z]\s*|
+ [Sa]\s*[FH]\s*[R]\s*[\x30U]\s*[Uc]\s*[D]\s*o\s*v\s*L\s*[\x2b\x2f-\x39w-z]\s*|
+ [Sa]\s*[FH]\s*[R]\s*[\x30U]\s*[Uc]\s*[FH]\s*[M]\s*\x36\s*L\s*y\s*[\x2b\x2f\x38-\x39]\s*
+ )
+ # Up to 260 characters (pre-encoding, reasonable URL length).
+ [A-Za-z0-9+/=\s]{1,357}
+ )
+ (?=[^A-Za-z0-9+/=\s]|$)
+ """, re.VERBOSE)
+
+# Get some valid obfuscated ip addresses.
+IPV4 = re.compile(r"""
+ (?:^|
+ (?![^\d\.])
+ )
+ (?:
+ (?:[1-9]?\d|1\d\d|2[0-4]\d|25[0-5])
+ [\[\(\\]*?\.[\]\)]*?
+ ){3}
+ (?:[1-9]?\d|1\d\d|2[0-4]\d|25[0-5])
+ (?:(?=[^\d\.])|$)
+ """, re.VERBOSE)
+
+# Experimental IPv6 regex, will not catch everything but should be sufficent for now.
+IPV6 = re.compile(r"""
+ \b(?:[a-f0-9]{1,4}:|:){2,7}(?:[a-f0-9]{1,4}|:)\b
+ """, re.IGNORECASE | re.VERBOSE)
+
+# Capture email addresses including common defangs.
+EMAIL = re.compile(r"""
+ (
+ [a-z0-9_.+-]+
+ [\(\[{\x20]*
+ (?:@|\Wat\W)
+ [\)\]}\x20]*
+ [a-z0-9-]+
+ (?:
+ (?:
+ (?:
+ \x20*
+ """ + SEPARATOR_DEFANGS + r"""
+ \x20*
+ )*
+ \.
+ (?:
+ \x20*
+ """ + SEPARATOR_DEFANGS + r"""
+ \x20*
+ )*
+ |
+ \W+dot\W+
+ )
+ [a-z0-9-]+?
+ )+
+ )
+ """ + END_PUNCTUATION + r"""
+ (?=\s|$)
+ """, re.IGNORECASE | re.VERBOSE | re.UNICODE)
+
+MD5 = re.compile(r"(?:[^a-fA-F\d]|\b)([a-fA-F\d]{32})(?:[^a-fA-F\d]|\b)")
+SHA1 = re.compile(r"(?:[^a-fA-F\d]|\b)([a-fA-F\d]{40})(?:[^a-fA-F\d]|\b)")
+SHA256 = re.compile(r"(?:[^a-fA-F\d]|\b)([a-fA-F\d]{64})(?:[^a-fA-F\d]|\b)")
+SHA512 = re.compile(
+ r"(?:[^a-fA-F\d]|\b)([a-fA-F\d]{128})(?:[^a-fA-F\d]|\b)")
+
+# YARA regex.
+YARA_PARSE = re.compile(r"""
+ (?:^|\s)
+ (
+ (?:
+ \s*?import\s+?"[^\r\n]*?[\r\n]+|
+ \s*?include\s+?"[^\r\n]*?[\r\n]+|
+ \s*?//[^\r\n]*[\r\n]+|
+ \s*?/\*.*?\*/\s*?
+ )*
+ (?:
+ \s*?private\s+|
+ \s*?global\s+
+ )*
+ rule\s*?
+ \w+\s*?
+ (?:
+ :[\s\w]+
+ )?
+ \s+\{
+ .*?
+ condition\s*?:
+ .*?
+ \s*\}
+ )
+ (?:$|\s)
+ """, re.MULTILINE | re.DOTALL | re.VERBOSE)
+
+CREDIT_CARD = re.compile(r"[0-9]{4}[ ]?[-]?[0-9]{4}[ ]?[-]?[0-9]{4}[ ]?[-]?[0-9]{4}")
+
+rintels = [(GENERIC_URL, "GENERIC_URL"),
+ (BRACKET_URL, "BRACKET_URL"),
+ (BACKSLASH_URL, "BACKSLASH_URL"),
+ (HEXENCODED_URL, "HEXENCODED_URL"),
+ (URLENCODED_URL, "URLENCODED_URL"),
+ (B64ENCODED_URL, "B64ENCODED_URL"),
+ (IPV4, "IPV4"),
+ (IPV6, "IPV6"),
+ (EMAIL, "EMAIL"),
+ (MD5, "MD5"),
+ (SHA1, "SHA1"),
+ (SHA256, "SHA256"),
+ (SHA512, "SHA512"),
+ (YARA_PARSE, "YARA_PARSE"),
+ (CREDIT_CARD, "CREDIT_CARD")]
+
+
+rscript = re.compile(r'<(script|SCRIPT).*(src|SRC)=([^\s>]+)')
+rhref = re.compile(r'<[aA].*(href|HREF)=([^\s>]+)')
+rendpoint = re.compile(r'[\'"](/.*?)[\'"]|[\'"](http.*?)[\'"]')
+rentropy = re.compile(r'[\w-]{16,45}')
diff --git a/core/lib/Photon/core/requester.py b/core/lib/Photon/core/requester.py
new file mode 100644
index 00000000..9711c18e
--- /dev/null
+++ b/core/lib/Photon/core/requester.py
@@ -0,0 +1,72 @@
+import random
+import time
+
+import requests
+from requests.exceptions import TooManyRedirects
+
+
+SESSION = requests.Session()
+SESSION.max_redirects = 3
+
+def requester(
+ url,
+ main_url=None,
+ delay=0,
+ cook=None,
+ headers=None,
+ timeout=10,
+ host=None,
+ proxies=[None],
+ user_agents=[None],
+ failed=None,
+ processed=None
+ ):
+ """Handle the requests and return the response body."""
+ cook = cook or set()
+ headers = headers or set()
+ user_agents = user_agents or ['Photon']
+ failed = failed or set()
+ processed = processed or set()
+ # Mark the URL as crawled
+ processed.add(url)
+ # Pause/sleep the program for specified time
+ time.sleep(delay)
+
+ def make_request(url):
+ """Default request"""
+ final_headers = headers or {
+ 'Host': host,
+ # Selecting a random user-agent
+ 'User-Agent': random.choice(user_agents),
+ 'Accept': 'text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8',
+ 'Accept-Language': 'en-US,en;q=0.5',
+ 'Accept-Encoding': 'gzip',
+ 'DNT': '1',
+ 'Connection': 'close',
+ }
+ try:
+ response = SESSION.get(
+ url,
+ cookies=cook,
+ headers=final_headers,
+ verify=False,
+ timeout=timeout,
+ stream=True,
+ proxies=random.choice(proxies)
+ )
+ except TooManyRedirects:
+ return 'dummy'
+
+ if 'text/html' in response.headers['content-type'] or \
+ 'text/plain' in response.headers['content-type']:
+ if response.status_code != '404':
+ return response.text
+ else:
+ response.close()
+ failed.add(url)
+ return 'dummy'
+ else:
+ response.close()
+ return 'dummy'
+
+ return make_request(url)
diff --git a/core/lib/Photon/core/updater.py b/core/lib/Photon/core/updater.py
new file mode 100644
index 00000000..3583923e
--- /dev/null
+++ b/core/lib/Photon/core/updater.py
@@ -0,0 +1,43 @@
+import os
+import re
+import sys
+from os import path
+sys.path.append(os.path.abspath('.'))
+
+from .colors import run, que, good, green, end, info
+from .requester import requester
+
+
+def updater():
+ """Update the current installation.
+
+ git clones the latest version and merges it with the current directory.
+ """
+ print('%s Checking for updates' % run)
+ # Changes must be separated by ;
+ changes = '''major bug fixes;removed ninja mode;dropped python < 3.2 support;fixed unicode output;proxy support;more intels'''
+ latest_commit = requester('https://raw.githubusercontent.com/s0md3v/Photon/master/core/updater.py', host='raw.githubusercontent.com')
+ # Just a hack to see if a new version is available
+ if changes not in latest_commit:
+ changelog = re.search(r"changes = '''(.*?)'''", latest_commit)
+ # Splitting the changes to form a list
+ changelog = changelog.group(1).split(';')
+ print('%s A new version of Photon is available.' % good)
+ print('%s Changes:' % info)
+ for change in changelog: # print changes
+ print('%s>%s %s' % (green, end, change))
+
+ current_path = os.getcwd().split('/') # if you know it, you know it
+ folder = current_path[-1] # current directory name
+ path = '/'.join(current_path) # current directory path
+ choice = input('%s Would you like to update? [Y/n] ' % que).lower()
+
+ if choice != 'n':
+ print('%s Updating Photon' % run)
+ os.system('git clone --quiet https://github.com/s0md3v/Photon %s'
+ % (folder))
+ os.system('cp -r %s/%s/* %s && rm -r %s/%s/ 2>/dev/null'
+ % (path, folder, path, path, folder))
+ print('%s Update successful!' % good)
+ else:
+ print('%s Photon is up to date!' % good)
diff --git a/core/lib/Photon/core/user-agents.txt b/core/lib/Photon/core/user-agents.txt
new file mode 100644
index 00000000..dc25d833
--- /dev/null
+++ b/core/lib/Photon/core/user-agents.txt
@@ -0,0 +1,18 @@
+Mozilla/4.0 (compatible; MSIE 6.0; MSIE 5.5; Windows NT 5.0) Opera 7.02 Bork-edition [en]
+Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.0; .NET CLR 1.0.3705)
+Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1; Trident/4.0; .NET CLR 2.0.50727; .NET CLR 3.0.4506.2152; .NET CLR 3.5.30729)
+Mozilla/5.0 (compatible; Baiduspider/2.0; +http://www.baidu.com/search/spider.html)
+Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; WOW64; Trident/5.0)
+Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_2) AppleWebKit/601.3.9 (KHTML, like Gecko) Version/9.0.2 Safari/601.3.9
+Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/42.0.2311.135 Safari/537.36 Edge/12.246
+Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; FSL 7.0.7.01001)
+Mozilla/5.0 (Windows NT 5.1; rv:13.0) Gecko/20100101 Firefox/13.0.1
+Mozilla/5.0 (Windows NT 6.0) AppleWebKit/535.1 (KHTML, like Gecko) Chrome/13.0.782.112 Safari/535.1
+Mozilla/5.0 (Windows NT 6.1; rv:5.0) Gecko/20100101 Firefox/5.02
+Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/47.0.2526.111 Safari/537.36
+Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1; SV1; .NET CLR 2.0.50727)
+Mozilla/5.0 (Windows NT 6.1; WOW64; rv:12.0) Gecko/20100101 Firefox/12.0
+Mozilla/5.0 (X11; CrOS x86_64 8172.45.0) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.64 Safari/537.36
+Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:15.0) Gecko/20100101 Firefox/15.0.1
+Mozilla/5.0 (X11; U; Linux x86_64; de; rv:1.9.2.8) Gecko/20100723 Ubuntu/10.04 (lucid) Firefox/3.6.8
+Opera/9.80 (Windows NT 5.1; U; en) Presto/2.10.289 Version/12.01
diff --git a/core/lib/Photon/core/utils.py b/core/lib/Photon/core/utils.py
new file mode 100644
index 00000000..3c6a7fe4
--- /dev/null
+++ b/core/lib/Photon/core/utils.py
@@ -0,0 +1,208 @@
+import requests
+import math
+import os.path
+import re
+import argparse
+import sys
+sys.path.append(os.path.abspath('.'))
+
+import tld
+
+from .colors import info
+from .config import VERBOSE, BAD_TYPES
+
+from urllib.parse import urlparse
+
+
+def regxy(pattern, response, supress_regex, custom):
+ """Extract a string based on regex pattern supplied by user."""
+ try:
+ matches = re.findall(r'%s' % pattern, response)
+ for match in matches:
+ verb('Custom regex', match)
+ custom.add(match)
+ except:
+ supress_regex = True
+
+
+def is_link(url, processed, files):
+ """
+ Determine whether or not a link should be crawled
+ A url should not be crawled if it
+ - Is a file
+ - Has already been crawled
+
+ Args:
+ url: str Url to be processed
+ processed: list[str] List of urls that have already been crawled
+
+ Returns:
+ bool If `url` should be crawled
+ """
+ if url not in processed:
+ if url.startswith('#') or url.startswith('javascript:'):
+ return False
+ is_file = url.endswith(BAD_TYPES)
+ if is_file:
+ files.add(url)
+ return False
+ return True
+ return False
+
+
+def remove_regex(urls, regex):
+ """
+ Parse a list for non-matches to a regex.
+
+ Args:
+ urls: iterable of urls
+ regex: string regex to be parsed for
+
+ Returns:
+ list of strings not matching regex
+ """
+
+ if not regex:
+ return urls
+
+ # To avoid iterating over the characters of a string
+ if not isinstance(urls, (list, set, tuple)):
+ urls = [urls]
+
+ try:
+ non_matching_urls = [url for url in urls if not re.search(regex, url)]
+ except TypeError:
+ return []
+
+ return non_matching_urls
+
+
+def writer(datasets, dataset_names, output_dir):
+ """Write the results."""
+ for dataset, dataset_name in zip(datasets, dataset_names):
+ if dataset:
+ filepath = output_dir + '/' + dataset_name + '.txt'
+ with open(filepath, 'w+') as out_file:
+ joined = '\n'.join(dataset)
+ out_file.write(str(joined.encode('utf-8').decode('utf-8')))
+ out_file.write('\n')
+
+
+def timer(diff, processed):
+ """Return the passed time."""
+ # Changes seconds into minutes and seconds
+ minutes, seconds = divmod(diff, 60)
+ try:
+ # Finds average time taken by requests
+ time_per_request = diff / float(len(processed))
+ except ZeroDivisionError:
+ time_per_request = 0
+ return minutes, seconds, time_per_request
+
+
+def entropy(string):
+ """Calculate the entropy of a string."""
+ entropy = 0
+ for number in range(256):
+ result = float(string.encode('utf-8').count(
+ chr(number))) / len(string.encode('utf-8'))
+ if result != 0:
+ entropy = entropy - result * math.log(result, 2)
+ return entropy
+
+
+def xml_parser(response):
+ """Extract links from .xml files."""
+ # Regex for extracting URLs
+ return re.findall(r'(.*?) ', response)
+
+
+def verb(kind, string):
+ """Enable verbose output."""
+ if VERBOSE:
+ print('%s %s: %s' % (info, kind, string))
+
+
+def extract_headers(headers):
+ """This function extracts valid headers from interactive input."""
+ sorted_headers = {}
+ matches = re.findall(r'(.*):\s(.*)', headers)
+ for match in matches:
+ header = match[0]
+ value = match[1]
+ try:
+ if value[-1] == ',':
+ value = value[:-1]
+ sorted_headers[header] = value
+ except IndexError:
+ pass
+ return sorted_headers
+
+
+def top_level(url, fix_protocol=True):
+ """Extract the top level domain from an URL."""
+ ext = tld.get_tld(url, fix_protocol=fix_protocol)
+ toplevel = '.'.join(urlparse(url).netloc.split('.')[-2:]).split(
+ ext)[0] + ext
+ return toplevel
+
+
+def is_proxy_list(v, proxies):
+ if os.path.isfile(v):
+ with open(v, 'r') as _file:
+ for line in _file:
+ line = line.strip()
+ if re.match(r"((http|socks5):\/\/.)?(\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}):(\d{1,5})", line) or \
+ re.match(r"((http|socks5):\/\/.)?[-a-zA-Z0-9@:%._\+~#=]{2,256}\.[a-z]{2,6}:(\d{1,5})", line):
+ proxies.append({"http": line,
+ "https": line})
+ else:
+ print("%s ignored" % line)
+ if proxies:
+ return True
+ return False
+
+
+def proxy_type(v):
+ """ Match IP:PORT or DOMAIN:PORT in a losse manner """
+ proxies = []
+ if re.match(r"((http|socks5):\/\/.)?(\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}):(\d{1,5})", v):
+ proxies.append({"http": v,
+ "https": v})
+ return proxies
+ elif re.match(r"((http|socks5):\/\/.)?[-a-zA-Z0-9@:%._\+~#=]{2,256}\.[a-z]{2,6}:(\d{1,5})", v):
+ proxies.append({"http": v,
+ "https": v})
+ return proxies
+ elif is_proxy_list(v, proxies):
+ return proxies
+ else:
+ raise argparse.ArgumentTypeError(
+ "Proxy should follow IP:PORT or DOMAIN:PORT format")
+
+
+def luhn(purported):
+
+ # sum_of_digits (index * 2)
+ LUHN_ODD_LOOKUP = (0, 2, 4, 6, 8, 1, 3, 5, 7, 9)
+
+ if not isinstance(purported, str):
+ purported = str(purported)
+ try:
+ evens = sum(int(p) for p in purported[-1::-2])
+ odds = sum(LUHN_ODD_LOOKUP[int(p)] for p in purported[-2::-2])
+ return (evens + odds) % 10 == 0
+ except ValueError: # Raised if an int conversion fails
+ return False
+
+
+def is_good_proxy(pip):
+ try:
+ requests.get('http://example.com/', proxies=pip, timeout=3)
+ except requests.exceptions.ConnectTimeout as e:
+ return False
+ except Exception as detail:
+ return False
+
+ return True
+
diff --git a/core/lib/Photon/core/zap.py b/core/lib/Photon/core/zap.py
new file mode 100644
index 00000000..8f1221b2
--- /dev/null
+++ b/core/lib/Photon/core/zap.py
@@ -0,0 +1,61 @@
+import re
+import requests
+import random
+import sys
+import os
+from os import path
+sys.path.append(os.path.abspath('.'))
+
+from .utils import verb, xml_parser
+from .colors import run, good
+from plugins.wayback import time_machine
+
+
+def zap(input_url, archive, domain, host, internal, robots, proxies):
+ """Extract links from robots.txt and sitemap.xml."""
+ if archive:
+ print('%s Fetching URLs from archive.org' % run)
+ if False:
+ archived_urls = time_machine(domain, 'domain')
+ else:
+ archived_urls = time_machine(host, 'host')
+ print('%s Retrieved %i URLs from archive.org' % (
+ good, len(archived_urls) - 1))
+ for url in archived_urls:
+ verb('Internal page', url)
+ internal.add(url)
+ # Makes request to robots.txt
+ response = requests.get(input_url + '/robots.txt',
+ proxies=random.choice(proxies)).text
+ # Making sure robots.txt isn't some fancy 404 page
+ if '(?s)', '', response)
+ res = re.sub(r'<[^<]+?>', '', res)
+ matches = rintel[0].findall(res)
+ if matches:
+ for match in matches:
+ verb('Intel', match)
+ bad_intel.add((match, rintel[1], url))
+
+
+def js_extractor(response):
+ """Extract js files from the response body"""
+ # Extract .js files
+ matches = rscript.findall(response)
+ for match in matches:
+ match = match[2].replace('\'', '').replace('"', '')
+ verb('JS file', match)
+ bad_scripts.add(match)
+
+def remove_file(url):
+ if url.count('/') > 2:
+ replacable = re.search(r'/[^/]*?$', url).group()
+ if replacable != '/':
+ return url.replace(replacable, '')
+ else:
+ return url
+ else:
+ return url
+
+def extractor(url):
+ """Extract details from the response body."""
+ response = requester(url, main_url, delay, cook, headers, timeout, host, proxies, user_agents, failed, processed)
+ if clone:
+ mirror(url, response)
+ matches = rhref.findall(response)
+ for link in matches:
+ # Remove everything after a "#" to deal with in-page anchors
+ link = link[1].replace('\'', '').replace('"', '').split('#')[0]
+ # Checks if the URLs should be crawled
+ if is_link(link, processed, files):
+ if link[:4] == 'http':
+ if link.startswith(main_url):
+ verb('Internal page', link)
+ internal.add(link)
+ else:
+ verb('External page', link)
+ external.add(link)
+ elif link[:2] == '//':
+ if link.split('/')[2].startswith(host):
+ verb('Internal page', link)
+ internal.add(schema + '://' + link)
+ else:
+ verb('External page', link)
+ external.add(link)
+ elif link[:1] == '/':
+ verb('Internal page', link)
+ internal.add(remove_file(url) + link)
+ else:
+ verb('Internal page', link)
+ usable_url = remove_file(url)
+ if usable_url.endswith('/'):
+ internal.add(usable_url + link)
+ elif link.startswith('/'):
+ internal.add(usable_url + link)
+ else:
+ internal.add(usable_url + '/' + link)
+
+ if not only_urls:
+ intel_extractor(url, response)
+ js_extractor(response)
+ if args.regex and not supress_regex:
+ regxy(args.regex, response, supress_regex, custom)
+ if api:
+ matches = rentropy.findall(response)
+ for match in matches:
+ if entropy(match) >= 4:
+ verb('Key', match)
+ keys.add(url + ': ' + match)
+
+
+def jscanner(url):
+ """Extract endpoints from JavaScript code."""
+ response = requester(url, main_url, delay, cook, headers, timeout, host, proxies, user_agents, failed, processed)
+ # Extract URLs/endpoints
+ matches = rendpoint.findall(response)
+ # Iterate over the matches, match is a tuple
+ for match in matches:
+ # Combining the items because one of them is always empty
+ match = match[0] + match[1]
+ # Making sure it's not some JavaScript code
+ if not re.search(r'[}{><"\']', match) and not match == '/':
+ verb('JS endpoint', match)
+ endpoints.add(match)
+
+
+# Records the time at which crawling started
+then = time.time()
+
+# Step 1. Extract urls from robots.txt & sitemap.xml
+zap(main_url, args.archive, domain, host, internal, robots, proxies)
+
+# This is so the level 1 emails are parsed as well
+internal = set(remove_regex(internal, args.exclude))
+
+# Step 2. Crawl recursively to the limit specified in "crawl_level"
+for level in range(crawl_level):
+ # Links to crawl = (all links - already crawled links) - links not to crawl
+ links = remove_regex(internal - processed, args.exclude)
+ # If links to crawl are 0 i.e. all links have been crawled
+ if not links:
+ break
+ # if crawled links are somehow more than all links. Possible? ;/
+ elif len(internal) <= len(processed):
+ if len(internal) > 2 + len(args.seeds):
+ break
+ print('%s Level %i: %i URLs' % (run, level + 1, len(links)))
+ try:
+ flash(extractor, links, thread_count)
+ except KeyboardInterrupt:
+ print('')
+ break
+
+if not only_urls:
+ for match in bad_scripts:
+ if match.startswith(main_url):
+ scripts.add(match)
+ elif match.startswith('/') and not match.startswith('//'):
+ scripts.add(main_url + match)
+ elif not match.startswith('http') and not match.startswith('//'):
+ scripts.add(main_url + '/' + match)
+ # Step 3. Scan the JavaScript files for endpoints
+ print('%s Crawling %i JavaScript files' % (run, len(scripts)))
+ flash(jscanner, scripts, thread_count)
+
+ for url in internal:
+ if '=' in url:
+ fuzzable.add(url)
+
+ for match, intel_name, url in bad_intel:
+ if isinstance(match, tuple):
+ for x in match: # Because "match" is a tuple
+ if x != '': # If the value isn't empty
+ if intel_name == "CREDIT_CARD":
+ if not luhn(match):
+ # garbage number
+ continue
+ intel.add("%s:%s" % (intel_name, x))
+ else:
+ if intel_name == "CREDIT_CARD":
+ if not luhn(match):
+ # garbage number
+ continue
+ intel.add("%s:%s:%s" % (url, intel_name, match))
+ for url in external:
+ try:
+ if top_level(url, fix_protocol=True) in INTELS:
+ intel.add(url)
+ except:
+ pass
+
+# Records the time at which crawling stopped
+now = time.time()
+# Finds total time taken
+diff = (now - then)
+minutes, seconds, time_per_request = timer(diff, processed)
+
+# Step 4. Save the results
+if not os.path.exists(output_dir): # if the directory doesn't exist
+ os.mkdir(output_dir) # create a new directory
+
+datasets = [files, intel, robots, custom, failed, internal, scripts,
+ external, fuzzable, endpoints, keys]
+dataset_names = ['files', 'intel', 'robots', 'custom', 'failed', 'internal',
+ 'scripts', 'external', 'fuzzable', 'endpoints', 'keys']
+
+writer(datasets, dataset_names, output_dir)
+# Printing out results
+print(('%s-%s' % (red, end)) * 50)
+for dataset, dataset_name in zip(datasets, dataset_names):
+ if dataset:
+ print('%s %s: %s' % (good, dataset_name.capitalize(), len(dataset)))
+print(('%s-%s' % (red, end)) * 50)
+
+print('%s Total requests made: %i' % (info, len(processed)))
+print('%s Total time taken: %i minutes %i seconds' % (info, minutes, seconds))
+print('%s Requests per second: %i' % (info, int(len(processed) / diff)))
+
+datasets = {
+ 'files': list(files), 'intel': list(intel), 'robots': list(robots),
+ 'custom': list(custom), 'failed': list(failed), 'internal': list(internal),
+ 'scripts': list(scripts), 'external': list(external),
+ 'fuzzable': list(fuzzable), 'endpoints': list(endpoints),
+ 'keys': list(keys)
+}
+
+if args.dns:
+ print('%s Enumerating subdomains' % run)
+ from plugins.find_subdomains import find_subdomains
+ subdomains = find_subdomains(domain)
+ print('%s %i subdomains found' % (info, len(subdomains)))
+ writer([subdomains], ['subdomains'], output_dir)
+ datasets['subdomains'] = subdomains
+ from plugins.dnsdumpster import dnsdumpster
+ print('%s Generating DNS map' % run)
+ dnsdumpster(domain, output_dir)
+
+if args.export:
+ from plugins.exporter import exporter
+ # exporter(directory, format, datasets)
+ exporter(output_dir, args.export, datasets)
+
+print('%s Results saved in %s%s%s directory' % (good, green, output_dir, end))
+
+if args.std:
+ for string in datasets[args.std]:
+ sys.stdout.write(string + '\n')
diff --git a/core/lib/Photon/plugins/__init__.py b/core/lib/Photon/plugins/__init__.py
new file mode 100644
index 00000000..2be64901
--- /dev/null
+++ b/core/lib/Photon/plugins/__init__.py
@@ -0,0 +1 @@
+"""Plugins for Photon."""
diff --git a/core/lib/Photon/plugins/dnsdumpster.py b/core/lib/Photon/plugins/dnsdumpster.py
new file mode 100644
index 00000000..29a9614a
--- /dev/null
+++ b/core/lib/Photon/plugins/dnsdumpster.py
@@ -0,0 +1,22 @@
+"""Support for dnsdumpster.com."""
+import re
+
+import requests
+
+
+def dnsdumpster(domain, output_dir):
+ """Query dnsdumpster.com."""
+ response = requests.Session().get('https://dnsdumpster.com/').text
+ csrf_token = re.search(
+ r"name=\"csrfmiddlewaretoken\" value=\"(.*?)\"", response).group(1)
+
+ cookies = {'csrftoken': csrf_token}
+ headers = {'Referer': 'https://dnsdumpster.com/'}
+ data = {'csrfmiddlewaretoken': csrf_token, 'targetip': domain}
+ response = requests.Session().post(
+ 'https://dnsdumpster.com/', cookies=cookies, data=data, headers=headers)
+
+ image = requests.get('https://dnsdumpster.com/static/map/%s.png' % domain)
+ if image.status_code == 200:
+ with open('%s/%s.png' % (output_dir, domain), 'wb') as f:
+ f.write(image.content)
diff --git a/core/lib/Photon/plugins/exporter.py b/core/lib/Photon/plugins/exporter.py
new file mode 100644
index 00000000..207cc674
--- /dev/null
+++ b/core/lib/Photon/plugins/exporter.py
@@ -0,0 +1,24 @@
+"""Support for exporting the results."""
+import csv
+import json
+
+
+def exporter(directory, method, datasets):
+ """Export the results."""
+ if method.lower() == 'json':
+ # Convert json_dict to a JSON styled string
+ json_string = json.dumps(datasets, indent=4)
+ savefile = open('{}/exported.json'.format(directory), 'w+')
+ savefile.write(json_string)
+ savefile.close()
+
+ if method.lower() == 'csv':
+ with open('{}/exported.csv'.format(directory), 'w+') as csvfile:
+ csv_writer = csv.writer(
+ csvfile, delimiter=',', quoting=csv.QUOTE_MINIMAL)
+ for key, values in datasets.items():
+ if values is None:
+ csv_writer.writerow([key])
+ else:
+ csv_writer.writerow([key] + values)
+ csvfile.close()
diff --git a/core/lib/Photon/plugins/find_subdomains.py b/core/lib/Photon/plugins/find_subdomains.py
new file mode 100644
index 00000000..98c9109a
--- /dev/null
+++ b/core/lib/Photon/plugins/find_subdomains.py
@@ -0,0 +1,14 @@
+"""Support for findsubdomains.com."""
+from re import findall
+
+from requests import get
+
+
+def find_subdomains(domain):
+ """Find subdomains according to the TLD."""
+ result = set()
+ response = get('https://findsubdomains.com/subdomains-of/' + domain).text
+ matches = findall(r'(?s)(.*?)
', response)
+ for match in matches:
+ result.add(match.replace(' ', '').replace('\n', ''))
+ return list(result)
diff --git a/core/lib/Photon/plugins/wayback.py b/core/lib/Photon/plugins/wayback.py
new file mode 100644
index 00000000..d15942f6
--- /dev/null
+++ b/core/lib/Photon/plugins/wayback.py
@@ -0,0 +1,22 @@
+"""Support for archive.org."""
+import datetime
+import json
+
+from requests import get
+
+
+def time_machine(host, mode):
+ """Query archive.org."""
+ now = datetime.datetime.now()
+ to = str(now.year) + str(now.day) + str(now.month)
+ if now.month > 6:
+ fro = str(now.year) + str(now.day) + str(now.month - 6)
+ else:
+ fro = str(now.year - 1) + str(now.day) + str(now.month + 6)
+ url = "http://web.archive.org/cdx/search?url=%s&matchType=%s&collapse=urlkey&fl=original&filter=mimetype:text/html&filter=statuscode:200&output=json&from=%s&to=%s" % (host, mode, fro, to)
+ response = get(url).text
+ parsed = json.loads(response)[1:]
+ urls = []
+ for item in parsed:
+ urls.append(item[0])
+ return urls
diff --git a/core/lib/Photon/requirements.txt b/core/lib/Photon/requirements.txt
new file mode 100644
index 00000000..19be6bc4
--- /dev/null
+++ b/core/lib/Photon/requirements.txt
@@ -0,0 +1,4 @@
+requests
+requests[socks]
+urllib3
+tld
\ No newline at end of file
diff --git a/core/lib/__pycache__/FileUtils.cpython-37.pyc b/core/lib/__pycache__/FileUtils.cpython-37.pyc
new file mode 100644
index 00000000..df4f5d53
Binary files /dev/null and b/core/lib/__pycache__/FileUtils.cpython-37.pyc differ
diff --git a/core/lib/dnsdump_mod/DNSDumpsterAPI.py b/core/lib/dnsdump_mod/DNSDumpsterAPI.py
new file mode 100644
index 00000000..685884ff
--- /dev/null
+++ b/core/lib/dnsdump_mod/DNSDumpsterAPI.py
@@ -0,0 +1,118 @@
+"""
+This is the (unofficial) Python API for dnsdumpster.com Website.
+Using this code, you can retrieve subdomains
+
+"""
+
+
+import requests
+import re
+import sys
+
+from bs4 import BeautifulSoup
+
+
+class DNSDumpsterAPI(object):
+
+ """DNSDumpsterAPI Main Handler"""
+
+ def __init__(self, verbose=False):
+ self.verbose = verbose
+
+ def display_message(self, s):
+ if self.verbose:
+ print('[verbose] %s' % s)
+
+
+ def retrieve_results(self, table):
+ res = []
+ trs = table.findAll('tr')
+ for tr in trs:
+ tds = tr.findAll('td')
+ pattern_ip = r'([0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3})'
+ ip = re.findall(pattern_ip, tds[1].text)[0]
+ domain = tds[0].text.replace('\n', '').split(' ')[0]
+ header = ' '.join(tds[0].text.replace('\n', '').split(' ')[1:])
+ reverse_dns = tds[1].find('span', attrs={}).text
+
+ additional_info = tds[2].text
+ country = tds[2].find('span', attrs={}).text
+ autonomous_system = additional_info.split(' ')[0]
+ provider = ' '.join(additional_info.split(' ')[1:])
+ provider = provider.replace(country, '')
+ data = {'domain': domain,
+ 'ip': ip,
+ 'reverse_dns': reverse_dns,
+ 'as': autonomous_system,
+ 'provider': provider,
+ 'country': country,
+ 'header': header}
+ res.append(data)
+ return res
+
+ def retrieve_txt_record(self, table):
+ res = []
+ for td in table.findAll('td'):
+ res.append(td.text)
+ return res
+
+
+ def search(self, domain):
+ dnsdumpster_url = 'https://dnsdumpster.com/'
+ s = requests.session()
+
+ req = s.get(dnsdumpster_url)
+ soup = BeautifulSoup(req.content, 'html.parser')
+ csrf_middleware = soup.findAll('input', attrs={'name': 'csrfmiddlewaretoken'})[0]['value']
+ self.display_message('Retrieved token: %s' % csrf_middleware)
+
+ cookies = {'csrftoken': csrf_middleware}
+ headers = {'Referer': dnsdumpster_url}
+ data = {'csrfmiddlewaretoken': csrf_middleware, 'targetip': domain}
+ req = s.post(dnsdumpster_url, cookies=cookies, data=data, headers=headers)
+
+ if req.status_code != 200:
+ print(
+ u"Unexpected status code from {url}: {code}".format(
+ url=dnsdumpster_url, code=req.status_code),
+ file=sys.stderr,
+ )
+ return []
+
+ if 'error' in req.content.decode('utf-8'):
+ print("There was an error getting results", file=sys.stderr)
+ return []
+
+ soup = BeautifulSoup(req.content, 'html.parser')
+ tables = soup.findAll('table')
+
+ res = {}
+ res['domain'] = domain
+ res['dns_records'] = {}
+ res['dns_records']['dns'] = self.retrieve_results(tables[0])
+ res['dns_records']['mx'] = self.retrieve_results(tables[1])
+ res['dns_records']['txt'] = self.retrieve_txt_record(tables[2])
+ res['dns_records']['host'] = self.retrieve_results(tables[3])
+
+ # Network mapping image
+ try:
+ val = soup.find('img', attrs={'class': 'img-responsive'})['src']
+ tmp_url = '{}{}'.format(dnsdumpster_url, val)
+ image_data = requests.get(tmp_url).content.encode('base64')
+ except:
+ image_data = None
+ finally:
+ res['image_data'] = image_data
+
+ # XLS hosts.
+ # eg. tsebo.com-201606131255.xlsx
+ try:
+ pattern = r'https://dnsdumpster.com/static/xls/' + domain + '-[0-9]{12}\.xlsx'
+ xls_url = re.findall(pattern, req.content)[0]
+ xls_data = requests.get(xls_url).content.encode('base64')
+ except:
+ xls_data = None
+ finally:
+ res['xls_data'] = xls_data
+
+ return res
diff --git a/core/lib/dnsdump_mod/DNSDumpsterAPI.pyc b/core/lib/dnsdump_mod/DNSDumpsterAPI.pyc
new file mode 100644
index 00000000..1ff1af6d
Binary files /dev/null and b/core/lib/dnsdump_mod/DNSDumpsterAPI.pyc differ
diff --git a/core/lib/dnsdump_mod/__init__.py b/core/lib/dnsdump_mod/__init__.py
new file mode 100644
index 00000000..d493f670
--- /dev/null
+++ b/core/lib/dnsdump_mod/__init__.py
@@ -0,0 +1,11 @@
+#!/usr/bin/env python
+# coding: utf-8
+
+#-:-:-:-:-:-:-:-:-:-:-:-:#
+# TIDoS Framework #
+#-:-:-:-:-:-:-:-:-:-:-:-:#
+
+#This module requires TIDoS Framework
+#https://github.com/0xInfection/TIDoS-Framework
+
+pass
diff --git a/core/lib/dnsdump_mod/__init__.pyc b/core/lib/dnsdump_mod/__init__.pyc
new file mode 100644
index 00000000..382a88d4
Binary files /dev/null and b/core/lib/dnsdump_mod/__init__.pyc differ
diff --git a/core/lib/dnsdump_mod/__pycache__/DNSDumpsterAPI.cpython-37.pyc b/core/lib/dnsdump_mod/__pycache__/DNSDumpsterAPI.cpython-37.pyc
new file mode 100644
index 00000000..949722c7
Binary files /dev/null and b/core/lib/dnsdump_mod/__pycache__/DNSDumpsterAPI.cpython-37.pyc differ
diff --git a/core/lib/dnsdump_mod/__pycache__/__init__.cpython-37.pyc b/core/lib/dnsdump_mod/__pycache__/__init__.cpython-37.pyc
new file mode 100644
index 00000000..57599102
Binary files /dev/null and b/core/lib/dnsdump_mod/__pycache__/__init__.cpython-37.pyc differ
diff --git a/lib/emailprotectionslib_mod/build/lib.linux-x86_64-2.7/emailprotectionslib/Resolver.py b/core/lib/emailprotectionslib/Resolver.py
similarity index 100%
rename from lib/emailprotectionslib_mod/build/lib.linux-x86_64-2.7/emailprotectionslib/Resolver.py
rename to core/lib/emailprotectionslib/Resolver.py
diff --git a/lib/emailprotectionslib_mod/build/lib.linux-x86_64-2.7/emailprotectionslib/__init__.py b/core/lib/emailprotectionslib/__init__.py
similarity index 100%
rename from lib/emailprotectionslib_mod/build/lib.linux-x86_64-2.7/emailprotectionslib/__init__.py
rename to core/lib/emailprotectionslib/__init__.py
diff --git a/core/lib/emailprotectionslib/__pycache__/Resolver.cpython-37.pyc b/core/lib/emailprotectionslib/__pycache__/Resolver.cpython-37.pyc
new file mode 100644
index 00000000..734904d8
Binary files /dev/null and b/core/lib/emailprotectionslib/__pycache__/Resolver.cpython-37.pyc differ
diff --git a/core/lib/emailprotectionslib/__pycache__/__init__.cpython-37.pyc b/core/lib/emailprotectionslib/__pycache__/__init__.cpython-37.pyc
new file mode 100644
index 00000000..66706854
Binary files /dev/null and b/core/lib/emailprotectionslib/__pycache__/__init__.cpython-37.pyc differ
diff --git a/core/lib/emailprotectionslib/__pycache__/dmarc.cpython-37.pyc b/core/lib/emailprotectionslib/__pycache__/dmarc.cpython-37.pyc
new file mode 100644
index 00000000..b8ee7b5c
Binary files /dev/null and b/core/lib/emailprotectionslib/__pycache__/dmarc.cpython-37.pyc differ
diff --git a/core/lib/emailprotectionslib/__pycache__/spf.cpython-37.pyc b/core/lib/emailprotectionslib/__pycache__/spf.cpython-37.pyc
new file mode 100644
index 00000000..6a3b8de5
Binary files /dev/null and b/core/lib/emailprotectionslib/__pycache__/spf.cpython-37.pyc differ
diff --git a/core/lib/emailprotectionslib/dmarc.py b/core/lib/emailprotectionslib/dmarc.py
new file mode 100644
index 00000000..29a13f94
--- /dev/null
+++ b/core/lib/emailprotectionslib/dmarc.py
@@ -0,0 +1,150 @@
+import re
+import logging
+import core.lib.emailprotectionslib.Resolver
+import tldextract
+
+
+class DmarcRecord(object):
+
+ def __init__(self, domain):
+ self.domain = domain
+ self.version = None
+ self.policy = None
+ self.pct = None
+ self.rua = None
+ self.ruf = None
+ self.subdomain_policy = None
+ self.dkim_alignment = None
+ self.spf_alignment = None
+ self.record = None
+
+ def __str__(self):
+ return self.record
+
+ def __eq__(self, other):
+ return self.__dict__ == other.__dict__
+
+ def _store_tag_data(self, tag_name, tag_value):
+ if tag_name == "v":
+ self.version = tag_value
+ elif tag_name == "p":
+ self.policy = tag_value
+ elif tag_name == "pct":
+ self.pct = tag_value
+ elif tag_name == "rua":
+ self.rua = tag_value
+ elif tag_name == "ruf":
+ self.ruf = tag_value
+ elif tag_name == "sp":
+ self.subdomain_policy = tag_value
+ elif tag_name == "adkim":
+ self.dkim_alignment = tag_value
+ elif tag_name == "aspf":
+ self.spf_alignment = tag_value
+
+ def process_tags(self, dmarc_string):
+ TAG_NAME, TAG_VALUE = (0, 1)
+ tags = _extract_tags(dmarc_string)
+ for tag in tags:
+ self._store_tag_data(tag[TAG_NAME], tag[TAG_VALUE])
+
+ def is_record_strong(self):
+ record_strong = False
+ if self.policy is not None and (self.policy == "reject" or self.policy == "quarantine"):
+ record_strong = True
+
+ if not record_strong:
+ try:
+ record_strong = self.is_org_domain_strong()
+ except OrgDomainException:
+ record_strong = False
+
+ return record_strong
+
+ def is_subdomain_policy_strong(self):
+ if self.subdomain_policy is not None:
+ return self.subdomain_policy == "reject" or self.subdomain_policy == "quarantine"
+
+ def is_org_domain_strong(self):
+ org_record = self.get_org_record()
+ subdomain_policy_strong = org_record.is_subdomain_policy_strong()
+ if subdomain_policy_strong is not None:
+ return subdomain_policy_strong
+ else:
+ return org_record.is_record_strong()
+
+ def get_org_record(self):
+ org_domain = self.get_org_domain()
+ if org_domain == self.domain:
+ raise OrgDomainException
+ else:
+ return DmarcRecord.from_domain(org_domain)
+
+ def get_org_domain(self):
+ try:
+ domain_parts = tldextract.extract(self.domain)
+ return "%(domain)s.%(tld)s" % {'domain': domain_parts.domain, 'tld': domain_parts.suffix}
+ except TypeError:
+ return None
+
+ @staticmethod
+ def from_dmarc_string(dmarc_string, domain):
+ if dmarc_string is not None:
+ dmarc_record = DmarcRecord(domain)
+ dmarc_record.record = dmarc_string
+ dmarc_record.process_tags(dmarc_string)
+ return dmarc_record
+ else:
+ return DmarcRecord(domain)
+
+ @staticmethod
+ def from_domain(domain):
+ dmarc_string = get_dmarc_string_for_domain(domain)
+ if dmarc_string is not None:
+ return DmarcRecord.from_dmarc_string(dmarc_string, domain)
+ else:
+ return DmarcRecord(domain)
+
+
+def _extract_tags(dmarc_record):
+ dmarc_pattern = "(\w+)=(.*?)(?:; ?|$)"
+ return re.findall(dmarc_pattern, dmarc_record)
+
+
+def _merge_txt_record_strings(txt_record):
+ # DMARC spec requires that TXT records containing multiple strings be cat'd together.
+ string_pattern = re.compile('"([^"]*)"')
+ txt_record_strings = string_pattern.findall(txt_record)
+ return "".join(txt_record_strings)
+
+
+def _match_dmarc_record(txt_record):
+ merged_txt_record = _merge_txt_record_strings(txt_record)
+ dmarc_pattern = re.compile('^(v=DMARC.*)')
+ potential_dmarc_match = dmarc_pattern.match(str(merged_txt_record))
+ return potential_dmarc_match
+
+
+def _find_record_from_answers(txt_records):
+ dmarc_record = None
+ for record in txt_records:
+ potential_match = _match_dmarc_record(record[2])
+ if potential_match is not None:
+ dmarc_record = potential_match.group(1)
+ return dmarc_record
+
+
+def get_dmarc_string_for_domain(domain):
+ try:
+ txt_records = Resolver.resolver().query("_dmarc." + domain, query_type="TXT")
+ return _find_record_from_answers(txt_records)
+ except IOError:
+ # This is returned usually as a NXDOMAIN, which is expected.
+ return None
+ except TypeError as error:
+ logging.exception(error)
+ return None
+
+
+class OrgDomainException(Exception):
+ pass
diff --git a/core/lib/emailprotectionslib/spf.py b/core/lib/emailprotectionslib/spf.py
new file mode 100644
index 00000000..e2c4d70d
--- /dev/null
+++ b/core/lib/emailprotectionslib/spf.py
@@ -0,0 +1,192 @@
+import re
+import logging
+import core.lib.emailprotectionslib.Resolver
+
+
+class SpfRecord(object):
+
+ def __init__(self, domain):
+ self.version = None
+ self.record = None
+ self.mechanisms = None
+ self.all_string = None
+ self.domain = domain
+ self.recursion_depth = 0
+
+ def __str__(self):
+ return self.record
+
+ def __eq__(self, other):
+ return self.__dict__ == other.__dict__
+
+ def get_redirected_record(self):
+ if self.recursion_depth >= 10:
+ return SpfRecord(self.get_redirect_domain())
+ else:
+ redirect_domain = self.get_redirect_domain()
+ if redirect_domain is not None:
+ redirect_record = SpfRecord.from_domain(redirect_domain)
+ redirect_record.recursion_depth = self.recursion_depth + 1
+ return redirect_record
+
+ def get_redirect_domain(self):
+ redirect_domain = None
+ if self.mechanisms is not None:
+ for mechanism in self.mechanisms:
+ redirect_mechanism = re.match('redirect=(.*)', mechanism)
+ if redirect_mechanism is not None:
+ redirect_domain = redirect_mechanism.group(1)
+ return redirect_domain
+
+ def get_include_domains(self):
+ include_domains = []
+ if self.mechanisms is not None:
+ for mechanism in self.mechanisms:
+ include_mechanism = re.match('include:(.*)', mechanism)
+ if include_mechanism is not None:
+ include_domains.append(include_mechanism.group(1))
+ return include_domains
+ else:
+ return []
+
+ def get_include_records(self):
+ if self.recursion_depth >= 10:
+ return {}
+ else:
+ include_domains = self.get_include_domains()
+ include_records = {}
+ for domain in include_domains:
+ try:
+ include_records[domain] = SpfRecord.from_domain(domain)
+ include_records[domain].recursion_depth = self.recursion_depth + 1
+ except IOError as e:
+ logging.exception(e)
+ include_records[domain] = None
+ return include_records
+
+ def _is_all_mechanism_strong(self):
+ strong_spf_all_string = True
+ if self.all_string is not None:
+ if not (self.all_string == "~all" or self.all_string == "-all"):
+ strong_spf_all_string = False
+ else:
+ strong_spf_all_string = False
+ return strong_spf_all_string
+
+ def _is_redirect_mechanism_strong(self):
+ redirect_domain = self.get_redirect_domain()
+
+ if redirect_domain is not None:
+ redirect_mechanism = SpfRecord.from_domain(redirect_domain)
+
+ if redirect_mechanism is not None:
+ return redirect_mechanism.is_record_strong()
+ else:
+ return False
+ else:
+ return False
+
+ def _are_include_mechanisms_strong(self):
+ include_records = self.get_include_records()
+ for record in include_records:
+ if include_records[record] is not None and include_records[record].is_record_strong():
+ return True
+ return False
+
+ def is_record_strong(self):
+ strong_spf_record = self._is_all_mechanism_strong()
+ if strong_spf_record is False:
+
+ redirect_strength = self._is_redirect_mechanism_strong()
+ include_strength = self._are_include_mechanisms_strong()
+
+ strong_spf_record = False
+
+ if redirect_strength is True:
+ strong_spf_record = True
+
+ if include_strength is True:
+ strong_spf_record = True
+ return strong_spf_record
+
+ @staticmethod
+ def from_spf_string(spf_string, domain):
+ if spf_string is not None:
+ spf_record = SpfRecord(domain)
+ spf_record.record = spf_string
+ spf_record.mechanisms = _extract_mechanisms(spf_string)
+ spf_record.version = _extract_version(spf_string)
+ spf_record.all_string = _extract_all_mechanism(spf_record.mechanisms)
+ return spf_record
+ else:
+ return SpfRecord(domain)
+
+ @staticmethod
+ def from_domain(domain):
+ spf_string = get_spf_string_for_domain(domain)
+ if spf_string is not None:
+ return SpfRecord.from_spf_string(spf_string, domain)
+ else:
+ return SpfRecord(domain)
+
+
+def _extract_version(spf_string):
+ version_pattern = "^v=(spf.)"
+ version_match = re.match(version_pattern, spf_string)
+ if version_match is not None:
+ return version_match.group(1)
+ else:
+ return None
+
+
+def _extract_all_mechanism(mechanisms):
+ all_mechanism = None
+ for mechanism in mechanisms:
+ if re.match(".all", mechanism):
+ all_mechanism = mechanism
+ return all_mechanism
+
+
+def _find_unique_mechanisms(initial_mechanisms, redirected_mechanisms):
+ return [x for x in redirected_mechanisms if x not in initial_mechanisms]
+
+
+def _extract_mechanisms(spf_string):
+ spf_mechanism_pattern = ("(?:((?:\+|-|~)?(?:a|mx|ptr|include"
+ "|ip4|ip6|exists|redirect|exp|all)"
+ "(?:(?::|=|/)?(?:\S*))?) ?)")
+ spf_mechanisms = re.findall(spf_mechanism_pattern, spf_string)
+
+ return spf_mechanisms
+
+
+def _merge_txt_record_strings(txt_record):
+ # SPF spec requires that TXT records containing multiple strings be cat'd together.
+ string_pattern = re.compile('"([^"]*)"')
+ txt_record_strings = string_pattern.findall(txt_record)
+ return "".join(txt_record_strings)
+
+
+def _match_spf_record(txt_record):
+ clean_txt_record = _merge_txt_record_strings(txt_record)
+ spf_pattern = re.compile('^(v=spf.*)')
+ potential_spf_match = spf_pattern.match(str(clean_txt_record))
+ return potential_spf_match
+
+
+def _find_record_from_answers(txt_records):
+ spf_record = None
+ for record in txt_records:
+ potential_match = _match_spf_record(record[2])
+ if potential_match is not None:
+ spf_record = potential_match.group(1)
+ return spf_record
+
+
+def get_spf_string_for_domain(domain):
+ try:
+ txt_records = Resolver.resolver().query(domain, query_type="TXT")
+ return _find_record_from_answers(txt_records)
+ except IOError as e:
+ # This is returned usually as a NXDOMAIN, which is expected.
+ return None
diff --git a/lib/mechanize_mod/build/lib.linux-x86_64-2.7/mechanize/__init__.py b/core/lib/mechanize/__init__.py
similarity index 100%
rename from lib/mechanize_mod/build/lib.linux-x86_64-2.7/mechanize/__init__.py
rename to core/lib/mechanize/__init__.py
diff --git a/core/lib/mechanize/__pycache__/__init__.cpython-37.pyc b/core/lib/mechanize/__pycache__/__init__.cpython-37.pyc
new file mode 100644
index 00000000..087638ec
Binary files /dev/null and b/core/lib/mechanize/__pycache__/__init__.cpython-37.pyc differ
diff --git a/core/lib/mechanize/__pycache__/_auth.cpython-37.pyc b/core/lib/mechanize/__pycache__/_auth.cpython-37.pyc
new file mode 100644
index 00000000..a8c0b601
Binary files /dev/null and b/core/lib/mechanize/__pycache__/_auth.cpython-37.pyc differ
diff --git a/core/lib/mechanize/__pycache__/_clientcookie.cpython-37.pyc b/core/lib/mechanize/__pycache__/_clientcookie.cpython-37.pyc
new file mode 100644
index 00000000..7d3e703b
Binary files /dev/null and b/core/lib/mechanize/__pycache__/_clientcookie.cpython-37.pyc differ
diff --git a/core/lib/mechanize/__pycache__/_debug.cpython-37.pyc b/core/lib/mechanize/__pycache__/_debug.cpython-37.pyc
new file mode 100644
index 00000000..c7d9a56d
Binary files /dev/null and b/core/lib/mechanize/__pycache__/_debug.cpython-37.pyc differ
diff --git a/core/lib/mechanize/__pycache__/_entities.cpython-37.pyc b/core/lib/mechanize/__pycache__/_entities.cpython-37.pyc
new file mode 100644
index 00000000..d424ad30
Binary files /dev/null and b/core/lib/mechanize/__pycache__/_entities.cpython-37.pyc differ
diff --git a/core/lib/mechanize/__pycache__/_equiv.cpython-37.pyc b/core/lib/mechanize/__pycache__/_equiv.cpython-37.pyc
new file mode 100644
index 00000000..3ecfd596
Binary files /dev/null and b/core/lib/mechanize/__pycache__/_equiv.cpython-37.pyc differ
diff --git a/core/lib/mechanize/__pycache__/_form.cpython-37.pyc b/core/lib/mechanize/__pycache__/_form.cpython-37.pyc
new file mode 100644
index 00000000..0b311a2d
Binary files /dev/null and b/core/lib/mechanize/__pycache__/_form.cpython-37.pyc differ
diff --git a/core/lib/mechanize/__pycache__/_form_controls.cpython-37.pyc b/core/lib/mechanize/__pycache__/_form_controls.cpython-37.pyc
new file mode 100644
index 00000000..4389b7da
Binary files /dev/null and b/core/lib/mechanize/__pycache__/_form_controls.cpython-37.pyc differ
diff --git a/core/lib/mechanize/__pycache__/_gzip.cpython-37.pyc b/core/lib/mechanize/__pycache__/_gzip.cpython-37.pyc
new file mode 100644
index 00000000..4bacbfa9
Binary files /dev/null and b/core/lib/mechanize/__pycache__/_gzip.cpython-37.pyc differ
diff --git a/core/lib/mechanize/__pycache__/_headersutil.cpython-37.pyc b/core/lib/mechanize/__pycache__/_headersutil.cpython-37.pyc
new file mode 100644
index 00000000..dfd08984
Binary files /dev/null and b/core/lib/mechanize/__pycache__/_headersutil.cpython-37.pyc differ
diff --git a/core/lib/mechanize/__pycache__/_html.cpython-37.pyc b/core/lib/mechanize/__pycache__/_html.cpython-37.pyc
new file mode 100644
index 00000000..f17b847f
Binary files /dev/null and b/core/lib/mechanize/__pycache__/_html.cpython-37.pyc differ
diff --git a/core/lib/mechanize/__pycache__/_http.cpython-37.pyc b/core/lib/mechanize/__pycache__/_http.cpython-37.pyc
new file mode 100644
index 00000000..80f60ed0
Binary files /dev/null and b/core/lib/mechanize/__pycache__/_http.cpython-37.pyc differ
diff --git a/core/lib/mechanize/__pycache__/_mechanize.cpython-37.pyc b/core/lib/mechanize/__pycache__/_mechanize.cpython-37.pyc
new file mode 100644
index 00000000..f878d1d6
Binary files /dev/null and b/core/lib/mechanize/__pycache__/_mechanize.cpython-37.pyc differ
diff --git a/core/lib/mechanize/__pycache__/_opener.cpython-37.pyc b/core/lib/mechanize/__pycache__/_opener.cpython-37.pyc
new file mode 100644
index 00000000..c3e94a7a
Binary files /dev/null and b/core/lib/mechanize/__pycache__/_opener.cpython-37.pyc differ
diff --git a/core/lib/mechanize/__pycache__/_request.cpython-37.pyc b/core/lib/mechanize/__pycache__/_request.cpython-37.pyc
new file mode 100644
index 00000000..562fd545
Binary files /dev/null and b/core/lib/mechanize/__pycache__/_request.cpython-37.pyc differ
diff --git a/core/lib/mechanize/__pycache__/_response.cpython-37.pyc b/core/lib/mechanize/__pycache__/_response.cpython-37.pyc
new file mode 100644
index 00000000..50ddeb47
Binary files /dev/null and b/core/lib/mechanize/__pycache__/_response.cpython-37.pyc differ
diff --git a/core/lib/mechanize/__pycache__/_rfc3986.cpython-37.pyc b/core/lib/mechanize/__pycache__/_rfc3986.cpython-37.pyc
new file mode 100644
index 00000000..453d0efb
Binary files /dev/null and b/core/lib/mechanize/__pycache__/_rfc3986.cpython-37.pyc differ
diff --git a/core/lib/mechanize/__pycache__/_sockettimeout.cpython-37.pyc b/core/lib/mechanize/__pycache__/_sockettimeout.cpython-37.pyc
new file mode 100644
index 00000000..152ad1ce
Binary files /dev/null and b/core/lib/mechanize/__pycache__/_sockettimeout.cpython-37.pyc differ
diff --git a/core/lib/mechanize/__pycache__/_urllib2.cpython-37.pyc b/core/lib/mechanize/__pycache__/_urllib2.cpython-37.pyc
new file mode 100644
index 00000000..42cbac8f
Binary files /dev/null and b/core/lib/mechanize/__pycache__/_urllib2.cpython-37.pyc differ
diff --git a/core/lib/mechanize/__pycache__/_urllib2_fork.cpython-37.pyc b/core/lib/mechanize/__pycache__/_urllib2_fork.cpython-37.pyc
new file mode 100644
index 00000000..2fae671f
Binary files /dev/null and b/core/lib/mechanize/__pycache__/_urllib2_fork.cpython-37.pyc differ
diff --git a/core/lib/mechanize/__pycache__/_useragent.cpython-37.pyc b/core/lib/mechanize/__pycache__/_useragent.cpython-37.pyc
new file mode 100644
index 00000000..69c2f585
Binary files /dev/null and b/core/lib/mechanize/__pycache__/_useragent.cpython-37.pyc differ
diff --git a/core/lib/mechanize/__pycache__/_util.cpython-37.pyc b/core/lib/mechanize/__pycache__/_util.cpython-37.pyc
new file mode 100644
index 00000000..d3e82356
Binary files /dev/null and b/core/lib/mechanize/__pycache__/_util.cpython-37.pyc differ
diff --git a/core/lib/mechanize/__pycache__/_version.cpython-37.pyc b/core/lib/mechanize/__pycache__/_version.cpython-37.pyc
new file mode 100644
index 00000000..53ecad65
Binary files /dev/null and b/core/lib/mechanize/__pycache__/_version.cpython-37.pyc differ
diff --git a/core/lib/mechanize/__pycache__/polyglot.cpython-37.pyc b/core/lib/mechanize/__pycache__/polyglot.cpython-37.pyc
new file mode 100644
index 00000000..231e1191
Binary files /dev/null and b/core/lib/mechanize/__pycache__/polyglot.cpython-37.pyc differ
diff --git a/core/lib/mechanize/_auth.py b/core/lib/mechanize/_auth.py
new file mode 100644
index 00000000..023a689b
--- /dev/null
+++ b/core/lib/mechanize/_auth.py
@@ -0,0 +1,73 @@
+"""HTTP Authentication and Proxy support.
+
+
+Copyright 2006 John J. Lee
+
+This code is free software; you can redistribute it and/or modify it under
+the terms of the BSD or ZPL 2.1 licenses (see the file LICENSE
+included with the distribution).
+
+"""
+
+from __future__ import absolute_import
+from ._urllib2_fork import HTTPPasswordMgr
+from .polyglot import is_string, iteritems
+
+
+# TODO: stop deriving from HTTPPasswordMgr
+class HTTPProxyPasswordMgr(HTTPPasswordMgr):
+ # has default realm and host/port
+
+ def add_password(self, realm, uri, user, passwd):
+ # uri could be a single URI or a sequence
+ if uri is None or is_string(uri):
+ uris = [uri]
+ else:
+ uris = uri
+ passwd_by_domain = self.passwd.setdefault(realm, {})
+ for uri in uris:
+ for default_port in True, False:
+ reduced_uri = self.reduce_uri(uri, default_port)
+ passwd_by_domain[reduced_uri] = (user, passwd)
+
+ def find_user_password(self, realm, authuri):
+ attempts = [(realm, authuri), (None, authuri)]
+ # bleh, want default realm to take precedence over default
+ # URI/authority, hence this outer loop
+ for default_uri in False, True:
+ for realm, authuri in attempts:
+ authinfo_by_domain = self.passwd.get(realm, {})
+ for default_port in True, False:
+ reduced_authuri = self.reduce_uri(authuri, default_port)
+ for uri, authinfo in iteritems(authinfo_by_domain):
+ if uri is None and not default_uri:
+ continue
+ if self.is_suburi(uri, reduced_authuri):
+ return authinfo
+ user, password = None, None
+
+ if user is not None:
+ break
+ return user, password
+
+ def reduce_uri(self, uri, default_port=True):
+ if uri is None:
+ return None
+ return HTTPPasswordMgr.reduce_uri(self, uri, default_port)
+
+ def is_suburi(self, base, test):
+ if base is None:
+ # default to the proxy's host/port
+ hostport, path = test
+ base = (hostport, "/")
+ return HTTPPasswordMgr.is_suburi(self, base, test)
+
+
+class HTTPSClientCertMgr(HTTPPasswordMgr):
+ # implementation inheritance: this is not a proper subclass
+
+ def add_key_cert(self, uri, key_file, cert_file):
+ self.add_password(None, uri, key_file, cert_file)
+
+ def find_key_cert(self, authuri):
+ return HTTPPasswordMgr.find_user_password(self, None, authuri)
diff --git a/core/lib/mechanize/_clientcookie.py b/core/lib/mechanize/_clientcookie.py
new file mode 100644
index 00000000..ebd27736
--- /dev/null
+++ b/core/lib/mechanize/_clientcookie.py
@@ -0,0 +1,252 @@
+from __future__ import absolute_import
+
+import re
+import time
+from .polyglot import (
+ Cookie as _Cookie, CookieJar as CJ, MozillaCookieJar as MCJ, request_host
+ as request_host_lc, DEFAULT_HTTP_PORT, CookiePolicy, DefaultCookiePolicy,
+ FileCookieJar, LoadError, LWPCookieJar, _debug, domain_match,
+ eff_request_host, escape_path, is_HDN, lwp_cookie_str, reach, request_path,
+ request_port, user_domain_match, iteritems)
+
+__all__ = [
+ 'DEFAULT_HTTP_PORT', 'CookiePolicy', 'DefaultCookiePolicy',
+ 'request_host_lc', 'MozillaCookieJar', 'escape_path', 'is_HDN',
+ 'request_port', 'LWPCookieJar', 'LoadError', 'reach', 'FileCookieJar',
+ 'lwp_cookie_str', 'domain_match', 'request_path', 'user_domain_match'
+]
+
+
+def effective_request_host(request):
+ """Return the effective request-host, as defined by RFC 2965."""
+ return eff_request_host(request)[1]
+
+
+def request_is_unverifiable(request):
+ try:
+ return request.is_unverifiable()
+ except AttributeError:
+ if hasattr(request, "unverifiable"):
+ return request.unverifiable
+ else:
+ raise
+
+
+def cookies_equal(a, b):
+ return all(getattr(a, x) == getattr(b, x) for x in Cookie._attrs)
+
+
+class Cookie(_Cookie):
+ _attrs = ("version", "name", "value", "port", "port_specified", "domain",
+ "domain_specified", "domain_initial_dot", "path",
+ "path_specified", "secure", "expires", "discard", "comment",
+ "comment_url", "rfc2109", "_rest")
+
+ def __eq__(self, other):
+ return all(getattr(self, a) == getattr(other, a) for a in self._attrs)
+
+ def __ne__(self, other):
+ return not (self == other)
+
+
+class CookieJar(CJ):
+
+ def __getstate__(self):
+ ans = self.__dict__.copy()
+ del ans['_cookies_lock']
+ return ans
+
+ def __setstate__(self, val):
+ for k, v in iteritems(val):
+ setattr(self, k, v)
+
+ def cookies_for_request(self, request):
+ """Return a list of cookies to be returned to server.
+
+ The returned list of cookie instances is sorted in the order they
+ should appear in the Cookie: header for return to the server.
+
+ See add_cookie_header.__doc__ for the interface required of the
+ request argument.
+ """
+ with self._cookies_lock:
+ self._policy._now = self._now = int(time.time())
+ cookies = self._cookies_for_request(request)
+
+ # add cookies in order of most specific (i.e. longest) path first
+ def key(x):
+ return len(x.path)
+
+ cookies.sort(key=key, reverse=True)
+ return cookies
+
+ def get_policy(self):
+ return self._policy
+
+ def _normalized_cookie_tuples(self, attrs_set):
+ """Return list of tuples containing normalised cookie information.
+
+ attrs_set is the list of lists of key,value pairs extracted from
+ the Set-Cookie or Set-Cookie2 headers.
+
+ Tuples are name, value, standard, rest, where name and value are the
+ cookie name and value, standard is a dictionary containing the standard
+ cookie-attributes (discard, secure, version, expires or max-age,
+ domain, path and port) and rest is a dictionary containing the rest of
+ the cookie-attributes.
+
+ """
+ cookie_tuples = []
+
+ boolean_attrs = "discard", "secure"
+ value_attrs = ("version", "expires", "max-age", "domain", "path",
+ "port", "comment", "commenturl")
+
+ for cookie_attrs in attrs_set:
+ name, value = cookie_attrs[0]
+
+ # Build dictionary of standard cookie-attributes (standard) and
+ # dictionary of other cookie-attributes (rest).
+
+ # Note: expiry time is normalised to seconds since epoch. V0
+ # cookies should have the Expires cookie-attribute, and V1 cookies
+ # should have Max-Age, but since V1 includes RFC 2109 cookies (and
+ # since V0 cookies may be a mish-mash of Netscape and RFC 2109), we
+ # accept either (but prefer Max-Age).
+ max_age_set = False
+
+ bad_cookie = False
+
+ standard = {}
+ rest = {}
+ for k, v in cookie_attrs[1:]:
+ lc = k.lower()
+ # don't lose case distinction for unknown fields
+ if lc in value_attrs or lc in boolean_attrs:
+ k = lc
+ if k in boolean_attrs and v is None:
+ # boolean cookie-attribute is present, but has no value
+ # (like "discard", rather than "port=80")
+ v = True
+ if k in standard:
+ # only first value is significant
+ continue
+ if k == "domain":
+ if v is None:
+ _debug(" missing value for domain attribute")
+ bad_cookie = True
+ break
+ # RFC 2965 section 3.3.3
+ v = v.lower()
+ if k == "expires":
+ if max_age_set:
+ # Prefer max-age to expires (like Mozilla)
+ continue
+ if v is None:
+ _debug(" missing or invalid value for expires "
+ "attribute: treating as session cookie")
+ continue
+ if k == "max-age":
+ max_age_set = True
+ try:
+ v = int(v)
+ except ValueError:
+ _debug(" missing or invalid (non-numeric) value for "
+ "max-age attribute")
+ bad_cookie = True
+ break
+ # convert RFC 2965 Max-Age to seconds since epoch
+ # XXX Strictly you're supposed to follow RFC 2616
+ # age-calculation rules. Remember that zero Max-Age
+ # is a request to discard (old and new) cookie, though.
+ k = "expires"
+ v = self._now + v
+ if not v and k == 'path':
+ # Added by Kovid, not in stdlib implementation
+ v = '/'
+ if (k in value_attrs) or (k in boolean_attrs):
+ if (v is None and
+ k not in ("port", "comment", "commenturl")):
+ _debug(" missing value for %s attribute" % k)
+ bad_cookie = True
+ break
+ standard[k] = v
+ else:
+ rest[k] = v
+
+ if bad_cookie:
+ continue
+
+ cookie_tuples.append((name, value, standard, rest))
+
+ return cookie_tuples
+
+ def __getitem__(self, i):
+ for q, ans in enumerate(self):
+ if q == i:
+ return ans
+ raise IndexError()
+
+
+class MozillaCookieJar(MCJ):
+
+ def _really_load(self, f, filename, ignore_discard, ignore_expires):
+ now = time.time()
+
+ magic = f.readline()
+ if not re.search(self.magic_re, magic):
+ f.close()
+ raise LoadError(
+ "%r does not look like a Netscape format cookies file" %
+ filename)
+
+ try:
+ while 1:
+ line = f.readline()
+ if line == "":
+ break
+
+ # last field may be absent, so keep any trailing tab
+ if line.endswith("\n"):
+ line = line[:-1]
+
+ # skip comments and blank lines XXX what is $ for?
+ if (line.strip().startswith(("#", "$")) or line.strip() == ""):
+ continue
+
+ (domain, domain_specified, path, secure, expires, name,
+ value) = line.split("\t", 6) # Changed by Kovid
+ secure = (secure == "TRUE")
+ domain_specified = (domain_specified == "TRUE")
+ if name == "":
+ # cookies.txt regards 'Set-Cookie: foo' as a cookie
+ # with no name, whereas cookielib regards it as a
+ # cookie with no value.
+ name = value
+ value = None
+
+ initial_dot = domain.startswith(".")
+ if domain_specified != initial_dot:
+ raise ValueError()
+ assert domain_specified == initial_dot
+
+ discard = False
+ if expires == "":
+ expires = None
+ discard = True
+
+ # assume path_specified is false
+ c = Cookie(0, name, value, None, False, domain,
+ domain_specified, initial_dot, path, False, secure,
+ expires, discard, None, None, {})
+ if not ignore_discard and c.discard:
+ continue
+ if not ignore_expires and c.is_expired(now):
+ continue
+ self.set_cookie(c)
+
+ except IOError:
+ raise
+ except Exception:
+ raise LoadError("invalid Netscape format cookies file %r: %r" %
+ (filename, line))
diff --git a/lib/mechanize_mod/build/lib.linux-x86_64-2.7/mechanize/_debug.py b/core/lib/mechanize/_debug.py
similarity index 100%
rename from lib/mechanize_mod/build/lib.linux-x86_64-2.7/mechanize/_debug.py
rename to core/lib/mechanize/_debug.py
diff --git a/core/lib/mechanize/_entities.py b/core/lib/mechanize/_entities.py
new file mode 100644
index 00000000..50a075cc
--- /dev/null
+++ b/core/lib/mechanize/_entities.py
@@ -0,0 +1,1751 @@
+#!/usr/bin/env python
+# vim:fileencoding=utf-8
+# Copyright: 2017, Kovid Goyal
+
+from __future__ import (absolute_import, division, print_function,
+ unicode_literals)
+
+html5_entities = {
+# ENTITY_DATA {{{
+ 'aacute': 'á',
+ 'abreve': 'ă',
+ 'ac': '∾',
+ 'acd': '∿',
+ 'ace': '∾̳',
+ 'acirc': 'â',
+ 'acute': '´',
+ 'acy': 'а',
+ 'aelig': 'æ',
+ 'af': '\u2061',
+ 'afr': '𝔞',
+ 'agrave': 'à',
+ 'alefsym': 'ℵ',
+ 'aleph': 'ℵ',
+ 'alpha': 'α',
+ 'amacr': 'ā',
+ 'amalg': '⨿',
+ 'amp': '&',
+ 'and': '∧',
+ 'andand': '⩕',
+ 'andd': '⩜',
+ 'andslope': '⩘',
+ 'andv': '⩚',
+ 'ang': '∠',
+ 'ange': '⦤',
+ 'angle': '∠',
+ 'angmsd': '∡',
+ 'angmsdaa': '⦨',
+ 'angmsdab': '⦩',
+ 'angmsdac': '⦪',
+ 'angmsdad': '⦫',
+ 'angmsdae': '⦬',
+ 'angmsdaf': '⦭',
+ 'angmsdag': '⦮',
+ 'angmsdah': '⦯',
+ 'angrt': '∟',
+ 'angrtvb': '⊾',
+ 'angrtvbd': '⦝',
+ 'angsph': '∢',
+ 'angst': 'Å',
+ 'angzarr': '⍼',
+ 'aogon': 'ą',
+ 'aopf': '𝕒',
+ 'ap': '≈',
+ 'apacir': '⩯',
+ 'ape': '≊',
+ 'apid': '≋',
+ 'apos': "'",
+ 'applyfunction': '\u2061',
+ 'approx': '≈',
+ 'approxeq': '≊',
+ 'aring': 'å',
+ 'ascr': '𝒶',
+ 'assign': '≔',
+ 'ast': '*',
+ 'asymp': '≈',
+ 'asympeq': '≍',
+ 'atilde': 'ã',
+ 'auml': 'ä',
+ 'awconint': '∳',
+ 'awint': '⨑',
+ 'backcong': '≌',
+ 'backepsilon': '϶',
+ 'backprime': '‵',
+ 'backsim': '∽',
+ 'backsimeq': '⋍',
+ 'backslash': '∖',
+ 'barv': '⫧',
+ 'barvee': '⊽',
+ 'barwed': '⌅',
+ 'barwedge': '⌅',
+ 'bbrk': '⎵',
+ 'bbrktbrk': '⎶',
+ 'bcong': '≌',
+ 'bcy': 'б',
+ 'bdquo': '„',
+ 'becaus': '∵',
+ 'because': '∵',
+ 'bemptyv': '⦰',
+ 'bepsi': '϶',
+ 'bernou': 'ℬ',
+ 'bernoullis': 'ℬ',
+ 'beta': 'β',
+ 'beth': 'ℶ',
+ 'between': '≬',
+ 'bfr': '𝔟',
+ 'bigcap': '⋂',
+ 'bigcirc': '◯',
+ 'bigcup': '⋃',
+ 'bigodot': '⨀',
+ 'bigoplus': '⨁',
+ 'bigotimes': '⨂',
+ 'bigsqcup': '⨆',
+ 'bigstar': '★',
+ 'bigtriangledown': '▽',
+ 'bigtriangleup': '△',
+ 'biguplus': '⨄',
+ 'bigvee': '⋁',
+ 'bigwedge': '⋀',
+ 'bkarow': '⤍',
+ 'blacklozenge': '⧫',
+ 'blacksquare': '▪',
+ 'blacktriangle': '▴',
+ 'blacktriangledown': '▾',
+ 'blacktriangleleft': '◂',
+ 'blacktriangleright': '▸',
+ 'blank': '␣',
+ 'blk12': '▒',
+ 'blk14': '░',
+ 'blk34': '▓',
+ 'block': '█',
+ 'bne': '=⃥',
+ 'bnequiv': '≡⃥',
+ 'bnot': '⌐',
+ 'bopf': '𝕓',
+ 'bot': '⊥',
+ 'bottom': '⊥',
+ 'bowtie': '⋈',
+ 'boxbox': '⧉',
+ 'boxdl': '┐',
+ 'boxdr': '┌',
+ 'boxh': '─',
+ 'boxhd': '┬',
+ 'boxhu': '┴',
+ 'boxminus': '⊟',
+ 'boxplus': '⊞',
+ 'boxtimes': '⊠',
+ 'boxul': '┘',
+ 'boxur': '└',
+ 'boxv': '│',
+ 'boxvh': '┼',
+ 'boxvl': '┤',
+ 'boxvr': '├',
+ 'bprime': '‵',
+ 'breve': '˘',
+ 'brvbar': '¦',
+ 'bscr': '𝒷',
+ 'bsemi': '⁏',
+ 'bsim': '∽',
+ 'bsime': '⋍',
+ 'bsol': '\\',
+ 'bsolb': '⧅',
+ 'bsolhsub': '⟈',
+ 'bull': '•',
+ 'bullet': '•',
+ 'bump': '≎',
+ 'bumpe': '≏',
+ 'bumpeq': '≏',
+ 'cacute': 'ć',
+ 'cap': '∩',
+ 'capand': '⩄',
+ 'capbrcup': '⩉',
+ 'capcap': '⩋',
+ 'capcup': '⩇',
+ 'capdot': '⩀',
+ 'capitaldifferentiald': 'ⅅ',
+ 'caps': '∩︀',
+ 'caret': '⁁',
+ 'caron': 'ˇ',
+ 'cayleys': 'ℭ',
+ 'ccaps': '⩍',
+ 'ccaron': 'č',
+ 'ccedil': 'ç',
+ 'ccirc': 'ĉ',
+ 'cconint': '∰',
+ 'ccups': '⩌',
+ 'ccupssm': '⩐',
+ 'cdot': 'ċ',
+ 'cedil': '¸',
+ 'cedilla': '¸',
+ 'cemptyv': '⦲',
+ 'cent': '¢',
+ 'centerdot': '·',
+ 'cfr': '𝔠',
+ 'chcy': 'ч',
+ 'check': '✓',
+ 'checkmark': '✓',
+ 'chi': 'χ',
+ 'cir': '○',
+ 'circ': 'ˆ',
+ 'circeq': '≗',
+ 'circlearrowleft': '↺',
+ 'circlearrowright': '↻',
+ 'circledast': '⊛',
+ 'circledcirc': '⊚',
+ 'circleddash': '⊝',
+ 'circledot': '⊙',
+ 'circledr': '®',
+ 'circleds': 'Ⓢ',
+ 'circleminus': '⊖',
+ 'circleplus': '⊕',
+ 'circletimes': '⊗',
+ 'cire': '≗',
+ 'cirfnint': '⨐',
+ 'cirmid': '⫯',
+ 'cirscir': '⧂',
+ 'clockwisecontourintegral': '∲',
+ 'closecurlydoublequote': '”',
+ 'closecurlyquote': '’',
+ 'clubs': '♣',
+ 'clubsuit': '♣',
+ 'colon': ':',
+ 'colone': '≔',
+ 'coloneq': '≔',
+ 'comma': ',',
+ 'commat': '@',
+ 'comp': '∁',
+ 'compfn': '∘',
+ 'complement': '∁',
+ 'complexes': 'ℂ',
+ 'cong': '≅',
+ 'congdot': '⩭',
+ 'congruent': '≡',
+ 'conint': '∮',
+ 'contourintegral': '∮',
+ 'copf': '𝕔',
+ 'coprod': '∐',
+ 'coproduct': '∐',
+ 'copy': '©',
+ 'copysr': '℗',
+ 'counterclockwisecontourintegral': '∳',
+ 'crarr': '↵',
+ 'cross': '✗',
+ 'cscr': '𝒸',
+ 'csub': '⫏',
+ 'csube': '⫑',
+ 'csup': '⫐',
+ 'csupe': '⫒',
+ 'ctdot': '⋯',
+ 'cudarrl': '⤸',
+ 'cudarrr': '⤵',
+ 'cuepr': '⋞',
+ 'cuesc': '⋟',
+ 'cularr': '↶',
+ 'cularrp': '⤽',
+ 'cup': '∪',
+ 'cupbrcap': '⩈',
+ 'cupcap': '⩆',
+ 'cupcup': '⩊',
+ 'cupdot': '⊍',
+ 'cupor': '⩅',
+ 'cups': '∪︀',
+ 'curarr': '↷',
+ 'curarrm': '⤼',
+ 'curlyeqprec': '⋞',
+ 'curlyeqsucc': '⋟',
+ 'curlyvee': '⋎',
+ 'curlywedge': '⋏',
+ 'curren': '¤',
+ 'curvearrowleft': '↶',
+ 'curvearrowright': '↷',
+ 'cuvee': '⋎',
+ 'cuwed': '⋏',
+ 'cwconint': '∲',
+ 'cwint': '∱',
+ 'cylcty': '⌭',
+ 'dagger': '†',
+ 'daleth': 'ℸ',
+ 'darr': '↓',
+ 'dash': '‐',
+ 'dashv': '⊣',
+ 'dbkarow': '⤏',
+ 'dblac': '˝',
+ 'dcaron': 'ď',
+ 'dcy': 'д',
+ 'dd': 'ⅆ',
+ 'ddagger': '‡',
+ 'ddarr': '⇊',
+ 'ddotrahd': '⤑',
+ 'ddotseq': '⩷',
+ 'deg': '°',
+ 'del': '∇',
+ 'delta': 'δ',
+ 'demptyv': '⦱',
+ 'dfisht': '⥿',
+ 'dfr': '𝔡',
+ 'dhar': '⥥',
+ 'dharl': '⇃',
+ 'dharr': '⇂',
+ 'diacriticalacute': '´',
+ 'diacriticaldot': '˙',
+ 'diacriticaldoubleacute': '˝',
+ 'diacriticalgrave': '`',
+ 'diacriticaltilde': '˜',
+ 'diam': '⋄',
+ 'diamond': '⋄',
+ 'diamondsuit': '♦',
+ 'diams': '♦',
+ 'die': '¨',
+ 'differentiald': 'ⅆ',
+ 'digamma': 'ϝ',
+ 'disin': '⋲',
+ 'div': '÷',
+ 'divide': '÷',
+ 'divideontimes': '⋇',
+ 'divonx': '⋇',
+ 'djcy': 'ђ',
+ 'dlcorn': '⌞',
+ 'dlcrop': '⌍',
+ 'dollar': '$',
+ 'dopf': '𝕕',
+ 'dot': '˙',
+ 'dotdot': '⃜',
+ 'doteq': '≐',
+ 'doteqdot': '≑',
+ 'dotequal': '≐',
+ 'dotminus': '∸',
+ 'dotplus': '∔',
+ 'dotsquare': '⊡',
+ 'doublebarwedge': '⌆',
+ 'doublecontourintegral': '∯',
+ 'doubledot': '¨',
+ 'doubledownarrow': '⇓',
+ 'doubleleftarrow': '⇐',
+ 'doubleleftrightarrow': '⇔',
+ 'doublelefttee': '⫤',
+ 'doublelongleftarrow': '⟸',
+ 'doublelongleftrightarrow': '⟺',
+ 'doublelongrightarrow': '⟹',
+ 'doublerightarrow': '⇒',
+ 'doublerighttee': '⊨',
+ 'doubleuparrow': '⇑',
+ 'doubleupdownarrow': '⇕',
+ 'doubleverticalbar': '∥',
+ 'downarrow': '↓',
+ 'downarrowbar': '⤓',
+ 'downarrowuparrow': '⇵',
+ 'downbreve': '̑',
+ 'downdownarrows': '⇊',
+ 'downharpoonleft': '⇃',
+ 'downharpoonright': '⇂',
+ 'downleftrightvector': '⥐',
+ 'downleftteevector': '⥞',
+ 'downleftvector': '↽',
+ 'downleftvectorbar': '⥖',
+ 'downrightteevector': '⥟',
+ 'downrightvector': '⇁',
+ 'downrightvectorbar': '⥗',
+ 'downtee': '⊤',
+ 'downteearrow': '↧',
+ 'drbkarow': '⤐',
+ 'drcorn': '⌟',
+ 'drcrop': '⌌',
+ 'dscr': '𝒹',
+ 'dscy': 'ѕ',
+ 'dsol': '⧶',
+ 'dstrok': 'đ',
+ 'dtdot': '⋱',
+ 'dtri': '▿',
+ 'dtrif': '▾',
+ 'duarr': '⇵',
+ 'duhar': '⥯',
+ 'dwangle': '⦦',
+ 'dzcy': 'џ',
+ 'dzigrarr': '⟿',
+ 'eacute': 'é',
+ 'easter': '⩮',
+ 'ecaron': 'ě',
+ 'ecir': '≖',
+ 'ecirc': 'ê',
+ 'ecolon': '≕',
+ 'ecy': 'э',
+ 'eddot': '⩷',
+ 'edot': 'ė',
+ 'ee': 'ⅇ',
+ 'efdot': '≒',
+ 'efr': '𝔢',
+ 'eg': '⪚',
+ 'egrave': 'è',
+ 'egs': '⪖',
+ 'egsdot': '⪘',
+ 'el': '⪙',
+ 'element': '∈',
+ 'elinters': '⏧',
+ 'ell': 'ℓ',
+ 'els': '⪕',
+ 'elsdot': '⪗',
+ 'emacr': 'ē',
+ 'empty': '∅',
+ 'emptyset': '∅',
+ 'emptysmallsquare': '◻',
+ 'emptyv': '∅',
+ 'emptyverysmallsquare': '▫',
+ 'emsp': '\u2003',
+ 'emsp13': '\u2004',
+ 'emsp14': '\u2005',
+ 'eng': 'ŋ',
+ 'ensp': '\u2002',
+ 'eogon': 'ę',
+ 'eopf': '𝕖',
+ 'epar': '⋕',
+ 'eparsl': '⧣',
+ 'eplus': '⩱',
+ 'epsi': 'ε',
+ 'epsilon': 'ε',
+ 'epsiv': 'ϵ',
+ 'eqcirc': '≖',
+ 'eqcolon': '≕',
+ 'eqsim': '≂',
+ 'eqslantgtr': '⪖',
+ 'eqslantless': '⪕',
+ 'equal': '⩵',
+ 'equals': '=',
+ 'equaltilde': '≂',
+ 'equest': '≟',
+ 'equilibrium': '⇌',
+ 'equiv': '≡',
+ 'equivdd': '⩸',
+ 'eqvparsl': '⧥',
+ 'erarr': '⥱',
+ 'erdot': '≓',
+ 'escr': 'ℯ',
+ 'esdot': '≐',
+ 'esim': '≂',
+ 'eta': 'η',
+ 'eth': 'ð',
+ 'euml': 'ë',
+ 'euro': '€',
+ 'excl': '!',
+ 'exist': '∃',
+ 'exists': '∃',
+ 'expectation': 'ℰ',
+ 'exponentiale': 'ⅇ',
+ 'fallingdotseq': '≒',
+ 'fcy': 'ф',
+ 'female': '♀',
+ 'ffilig': 'ffi',
+ 'fflig': 'ff',
+ 'ffllig': 'ffl',
+ 'ffr': '𝔣',
+ 'filig': 'fi',
+ 'filledsmallsquare': '◼',
+ 'filledverysmallsquare': '▪',
+ 'fjlig': 'fj',
+ 'flat': '♭',
+ 'fllig': 'fl',
+ 'fltns': '▱',
+ 'fnof': 'ƒ',
+ 'fopf': '𝕗',
+ 'forall': '∀',
+ 'fork': '⋔',
+ 'forkv': '⫙',
+ 'fouriertrf': 'ℱ',
+ 'fpartint': '⨍',
+ 'frac12': '½',
+ 'frac13': '⅓',
+ 'frac14': '¼',
+ 'frac15': '⅕',
+ 'frac16': '⅙',
+ 'frac18': '⅛',
+ 'frac23': '⅔',
+ 'frac25': '⅖',
+ 'frac34': '¾',
+ 'frac35': '⅗',
+ 'frac38': '⅜',
+ 'frac45': '⅘',
+ 'frac56': '⅚',
+ 'frac58': '⅝',
+ 'frac78': '⅞',
+ 'frasl': '⁄',
+ 'frown': '⌢',
+ 'fscr': '𝒻',
+ 'gacute': 'ǵ',
+ 'gamma': 'γ',
+ 'gammad': 'ϝ',
+ 'gap': '⪆',
+ 'gbreve': 'ğ',
+ 'gcedil': 'Ģ',
+ 'gcirc': 'ĝ',
+ 'gcy': 'г',
+ 'gdot': 'ġ',
+ 'ge': '≥',
+ 'gel': '⋛',
+ 'geq': '≥',
+ 'geqq': '≧',
+ 'geqslant': '⩾',
+ 'ges': '⩾',
+ 'gescc': '⪩',
+ 'gesdot': '⪀',
+ 'gesdoto': '⪂',
+ 'gesdotol': '⪄',
+ 'gesl': '⋛︀',
+ 'gesles': '⪔',
+ 'gfr': '𝔤',
+ 'gg': '≫',
+ 'ggg': '⋙',
+ 'gimel': 'ℷ',
+ 'gjcy': 'ѓ',
+ 'gl': '≷',
+ 'gla': '⪥',
+ 'gle': '⪒',
+ 'glj': '⪤',
+ 'gnap': '⪊',
+ 'gnapprox': '⪊',
+ 'gne': '⪈',
+ 'gneq': '⪈',
+ 'gneqq': '≩',
+ 'gnsim': '⋧',
+ 'gopf': '𝕘',
+ 'grave': '`',
+ 'greaterequal': '≥',
+ 'greaterequalless': '⋛',
+ 'greaterfullequal': '≧',
+ 'greatergreater': '⪢',
+ 'greaterless': '≷',
+ 'greaterslantequal': '⩾',
+ 'greatertilde': '≳',
+ 'gscr': 'ℊ',
+ 'gsim': '≳',
+ 'gsime': '⪎',
+ 'gsiml': '⪐',
+ 'gt': '>',
+ 'gtcc': '⪧',
+ 'gtcir': '⩺',
+ 'gtdot': '⋗',
+ 'gtlpar': '⦕',
+ 'gtquest': '⩼',
+ 'gtrapprox': '⪆',
+ 'gtrarr': '⥸',
+ 'gtrdot': '⋗',
+ 'gtreqless': '⋛',
+ 'gtreqqless': '⪌',
+ 'gtrless': '≷',
+ 'gtrsim': '≳',
+ 'gvertneqq': '≩︀',
+ 'gvne': '≩︀',
+ 'hacek': 'ˇ',
+ 'hairsp': '\u200a',
+ 'half': '½',
+ 'hamilt': 'ℋ',
+ 'hardcy': 'ъ',
+ 'harr': '↔',
+ 'harrcir': '⥈',
+ 'harrw': '↭',
+ 'hat': '^',
+ 'hbar': 'ℏ',
+ 'hcirc': 'ĥ',
+ 'hearts': '♥',
+ 'heartsuit': '♥',
+ 'hellip': '…',
+ 'hercon': '⊹',
+ 'hfr': '𝔥',
+ 'hilbertspace': 'ℋ',
+ 'hksearow': '⤥',
+ 'hkswarow': '⤦',
+ 'hoarr': '⇿',
+ 'homtht': '∻',
+ 'hookleftarrow': '↩',
+ 'hookrightarrow': '↪',
+ 'hopf': '𝕙',
+ 'horbar': '―',
+ 'horizontalline': '─',
+ 'hscr': '𝒽',
+ 'hslash': 'ℏ',
+ 'hstrok': 'ħ',
+ 'humpdownhump': '≎',
+ 'humpequal': '≏',
+ 'hybull': '⁃',
+ 'hyphen': '‐',
+ 'iacute': 'í',
+ 'ic': '\u2063',
+ 'icirc': 'î',
+ 'icy': 'и',
+ 'idot': 'İ',
+ 'iecy': 'е',
+ 'iexcl': '¡',
+ 'iff': '⇔',
+ 'ifr': '𝔦',
+ 'igrave': 'ì',
+ 'ii': 'ⅈ',
+ 'iiiint': '⨌',
+ 'iiint': '∭',
+ 'iinfin': '⧜',
+ 'iiota': '℩',
+ 'ijlig': 'ij',
+ 'im': 'ℑ',
+ 'imacr': 'ī',
+ 'image': 'ℑ',
+ 'imaginaryi': 'ⅈ',
+ 'imagline': 'ℐ',
+ 'imagpart': 'ℑ',
+ 'imath': 'ı',
+ 'imof': '⊷',
+ 'imped': 'Ƶ',
+ 'implies': '⇒',
+ 'in': '∈',
+ 'incare': '℅',
+ 'infin': '∞',
+ 'infintie': '⧝',
+ 'inodot': 'ı',
+ 'int': '∫',
+ 'intcal': '⊺',
+ 'integers': 'ℤ',
+ 'integral': '∫',
+ 'intercal': '⊺',
+ 'intersection': '⋂',
+ 'intlarhk': '⨗',
+ 'intprod': '⨼',
+ 'invisiblecomma': '\u2063',
+ 'invisibletimes': '\u2062',
+ 'iocy': 'ё',
+ 'iogon': 'į',
+ 'iopf': '𝕚',
+ 'iota': 'ι',
+ 'iprod': '⨼',
+ 'iquest': '¿',
+ 'iscr': '𝒾',
+ 'isin': '∈',
+ 'isindot': '⋵',
+ 'isine': '⋹',
+ 'isins': '⋴',
+ 'isinsv': '⋳',
+ 'isinv': '∈',
+ 'it': '\u2062',
+ 'itilde': 'ĩ',
+ 'iukcy': 'і',
+ 'iuml': 'ï',
+ 'jcirc': 'ĵ',
+ 'jcy': 'й',
+ 'jfr': '𝔧',
+ 'jmath': 'ȷ',
+ 'jopf': '𝕛',
+ 'jscr': '𝒿',
+ 'jsercy': 'ј',
+ 'jukcy': 'є',
+ 'kappa': 'κ',
+ 'kappav': 'ϰ',
+ 'kcedil': 'ķ',
+ 'kcy': 'к',
+ 'kfr': '𝔨',
+ 'kgreen': 'ĸ',
+ 'khcy': 'х',
+ 'kjcy': 'ќ',
+ 'kopf': '𝕜',
+ 'kscr': '𝓀',
+ 'laarr': '⇚',
+ 'lacute': 'ĺ',
+ 'laemptyv': '⦴',
+ 'lagran': 'ℒ',
+ 'lambda': 'λ',
+ 'lang': '⟨',
+ 'langd': '⦑',
+ 'langle': '⟨',
+ 'lap': '⪅',
+ 'laplacetrf': 'ℒ',
+ 'laquo': '«',
+ 'larr': '←',
+ 'larrb': '⇤',
+ 'larrbfs': '⤟',
+ 'larrfs': '⤝',
+ 'larrhk': '↩',
+ 'larrlp': '↫',
+ 'larrpl': '⤹',
+ 'larrsim': '⥳',
+ 'larrtl': '↢',
+ 'lat': '⪫',
+ 'latail': '⤙',
+ 'late': '⪭',
+ 'lates': '⪭︀',
+ 'lbarr': '⤌',
+ 'lbbrk': '❲',
+ 'lbrace': '{',
+ 'lbrack': '[',
+ 'lbrke': '⦋',
+ 'lbrksld': '⦏',
+ 'lbrkslu': '⦍',
+ 'lcaron': 'ľ',
+ 'lcedil': 'ļ',
+ 'lceil': '⌈',
+ 'lcub': '{',
+ 'lcy': 'л',
+ 'ldca': '⤶',
+ 'ldquo': '“',
+ 'ldquor': '„',
+ 'ldrdhar': '⥧',
+ 'ldrushar': '⥋',
+ 'ldsh': '↲',
+ 'le': '≤',
+ 'leftanglebracket': '⟨',
+ 'leftarrow': '←',
+ 'leftarrowbar': '⇤',
+ 'leftarrowrightarrow': '⇆',
+ 'leftarrowtail': '↢',
+ 'leftceiling': '⌈',
+ 'leftdoublebracket': '⟦',
+ 'leftdownteevector': '⥡',
+ 'leftdownvector': '⇃',
+ 'leftdownvectorbar': '⥙',
+ 'leftfloor': '⌊',
+ 'leftharpoondown': '↽',
+ 'leftharpoonup': '↼',
+ 'leftleftarrows': '⇇',
+ 'leftrightarrow': '↔',
+ 'leftrightarrows': '⇆',
+ 'leftrightharpoons': '⇋',
+ 'leftrightsquigarrow': '↭',
+ 'leftrightvector': '⥎',
+ 'lefttee': '⊣',
+ 'leftteearrow': '↤',
+ 'leftteevector': '⥚',
+ 'leftthreetimes': '⋋',
+ 'lefttriangle': '⊲',
+ 'lefttrianglebar': '⧏',
+ 'lefttriangleequal': '⊴',
+ 'leftupdownvector': '⥑',
+ 'leftupteevector': '⥠',
+ 'leftupvector': '↿',
+ 'leftupvectorbar': '⥘',
+ 'leftvector': '↼',
+ 'leftvectorbar': '⥒',
+ 'leg': '⋚',
+ 'leq': '≤',
+ 'leqq': '≦',
+ 'leqslant': '⩽',
+ 'les': '⩽',
+ 'lescc': '⪨',
+ 'lesdot': '⩿',
+ 'lesdoto': '⪁',
+ 'lesdotor': '⪃',
+ 'lesg': '⋚︀',
+ 'lesges': '⪓',
+ 'lessapprox': '⪅',
+ 'lessdot': '⋖',
+ 'lesseqgtr': '⋚',
+ 'lesseqqgtr': '⪋',
+ 'lessequalgreater': '⋚',
+ 'lessfullequal': '≦',
+ 'lessgreater': '≶',
+ 'lessgtr': '≶',
+ 'lessless': '⪡',
+ 'lesssim': '≲',
+ 'lessslantequal': '⩽',
+ 'lesstilde': '≲',
+ 'lfisht': '⥼',
+ 'lfloor': '⌊',
+ 'lfr': '𝔩',
+ 'lg': '≶',
+ 'lge': '⪑',
+ 'lhar': '⥢',
+ 'lhard': '↽',
+ 'lharu': '↼',
+ 'lharul': '⥪',
+ 'lhblk': '▄',
+ 'ljcy': 'љ',
+ 'll': '≪',
+ 'llarr': '⇇',
+ 'llcorner': '⌞',
+ 'lleftarrow': '⇚',
+ 'llhard': '⥫',
+ 'lltri': '◺',
+ 'lmidot': 'ŀ',
+ 'lmoust': '⎰',
+ 'lmoustache': '⎰',
+ 'lnap': '⪉',
+ 'lnapprox': '⪉',
+ 'lne': '⪇',
+ 'lneq': '⪇',
+ 'lneqq': '≨',
+ 'lnsim': '⋦',
+ 'loang': '⟬',
+ 'loarr': '⇽',
+ 'lobrk': '⟦',
+ 'longleftarrow': '⟵',
+ 'longleftrightarrow': '⟷',
+ 'longmapsto': '⟼',
+ 'longrightarrow': '⟶',
+ 'looparrowleft': '↫',
+ 'looparrowright': '↬',
+ 'lopar': '⦅',
+ 'lopf': '𝕝',
+ 'loplus': '⨭',
+ 'lotimes': '⨴',
+ 'lowast': '∗',
+ 'lowbar': '_',
+ 'lowerleftarrow': '↙',
+ 'lowerrightarrow': '↘',
+ 'loz': '◊',
+ 'lozenge': '◊',
+ 'lozf': '⧫',
+ 'lpar': '(',
+ 'lparlt': '⦓',
+ 'lrarr': '⇆',
+ 'lrcorner': '⌟',
+ 'lrhar': '⇋',
+ 'lrhard': '⥭',
+ 'lrm': '\u200e',
+ 'lrtri': '⊿',
+ 'lsaquo': '‹',
+ 'lscr': '𝓁',
+ 'lsh': '↰',
+ 'lsim': '≲',
+ 'lsime': '⪍',
+ 'lsimg': '⪏',
+ 'lsqb': '[',
+ 'lsquo': '‘',
+ 'lsquor': '‚',
+ 'lstrok': 'ł',
+ 'lt': '<',
+ 'ltcc': '⪦',
+ 'ltcir': '⩹',
+ 'ltdot': '⋖',
+ 'lthree': '⋋',
+ 'ltimes': '⋉',
+ 'ltlarr': '⥶',
+ 'ltquest': '⩻',
+ 'ltri': '◃',
+ 'ltrie': '⊴',
+ 'ltrif': '◂',
+ 'ltrpar': '⦖',
+ 'lurdshar': '⥊',
+ 'luruhar': '⥦',
+ 'lvertneqq': '≨︀',
+ 'lvne': '≨︀',
+ 'macr': '¯',
+ 'male': '♂',
+ 'malt': '✠',
+ 'maltese': '✠',
+ 'map': '↦',
+ 'mapsto': '↦',
+ 'mapstodown': '↧',
+ 'mapstoleft': '↤',
+ 'mapstoup': '↥',
+ 'marker': '▮',
+ 'mcomma': '⨩',
+ 'mcy': 'м',
+ 'mdash': '—',
+ 'mddot': '∺',
+ 'measuredangle': '∡',
+ 'mediumspace': '\u205f',
+ 'mellintrf': 'ℳ',
+ 'mfr': '𝔪',
+ 'mho': '℧',
+ 'micro': 'µ',
+ 'mid': '∣',
+ 'midast': '*',
+ 'midcir': '⫰',
+ 'middot': '·',
+ 'minus': '−',
+ 'minusb': '⊟',
+ 'minusd': '∸',
+ 'minusdu': '⨪',
+ 'minusplus': '∓',
+ 'mlcp': '⫛',
+ 'mldr': '…',
+ 'mnplus': '∓',
+ 'models': '⊧',
+ 'mopf': '𝕞',
+ 'mp': '∓',
+ 'mscr': '𝓂',
+ 'mstpos': '∾',
+ 'mu': 'μ',
+ 'multimap': '⊸',
+ 'mumap': '⊸',
+ 'nabla': '∇',
+ 'nacute': 'ń',
+ 'nang': '∠⃒',
+ 'nap': '≉',
+ 'nape': '⩰̸',
+ 'napid': '≋̸',
+ 'napos': 'ʼn',
+ 'napprox': '≉',
+ 'natur': '♮',
+ 'natural': '♮',
+ 'naturals': 'ℕ',
+ 'nbsp': '\xa0',
+ 'nbump': '≎̸',
+ 'nbumpe': '≏̸',
+ 'ncap': '⩃',
+ 'ncaron': 'ň',
+ 'ncedil': 'ņ',
+ 'ncong': '≇',
+ 'ncongdot': '⩭̸',
+ 'ncup': '⩂',
+ 'ncy': 'н',
+ 'ndash': '–',
+ 'ne': '≠',
+ 'nearhk': '⤤',
+ 'nearr': '↗',
+ 'nearrow': '↗',
+ 'nedot': '≐̸',
+ 'negativemediumspace': '\u200b',
+ 'negativethickspace': '\u200b',
+ 'negativethinspace': '\u200b',
+ 'negativeverythinspace': '\u200b',
+ 'nequiv': '≢',
+ 'nesear': '⤨',
+ 'nesim': '≂̸',
+ 'nestedgreatergreater': '≫',
+ 'nestedlessless': '≪',
+ 'newline': '\n',
+ 'nexist': '∄',
+ 'nexists': '∄',
+ 'nfr': '𝔫',
+ 'nge': '≱',
+ 'ngeq': '≱',
+ 'ngeqq': '≧̸',
+ 'ngeqslant': '⩾̸',
+ 'nges': '⩾̸',
+ 'ngg': '⋙̸',
+ 'ngsim': '≵',
+ 'ngt': '≯',
+ 'ngtr': '≯',
+ 'ngtv': '≫̸',
+ 'nharr': '↮',
+ 'nhpar': '⫲',
+ 'ni': '∋',
+ 'nis': '⋼',
+ 'nisd': '⋺',
+ 'niv': '∋',
+ 'njcy': 'њ',
+ 'nlarr': '↚',
+ 'nldr': '‥',
+ 'nle': '≰',
+ 'nleftarrow': '↚',
+ 'nleftrightarrow': '↮',
+ 'nleq': '≰',
+ 'nleqq': '≦̸',
+ 'nleqslant': '⩽̸',
+ 'nles': '⩽̸',
+ 'nless': '≮',
+ 'nll': '⋘̸',
+ 'nlsim': '≴',
+ 'nlt': '≮',
+ 'nltri': '⋪',
+ 'nltrie': '⋬',
+ 'nltv': '≪̸',
+ 'nmid': '∤',
+ 'nobreak': '\u2060',
+ 'nonbreakingspace': '\xa0',
+ 'nopf': '𝕟',
+ 'not': '¬',
+ 'notcongruent': '≢',
+ 'notcupcap': '≭',
+ 'notdoubleverticalbar': '∦',
+ 'notelement': '∉',
+ 'notequal': '≠',
+ 'notequaltilde': '≂̸',
+ 'notexists': '∄',
+ 'notgreater': '≯',
+ 'notgreaterequal': '≱',
+ 'notgreaterfullequal': '≧̸',
+ 'notgreatergreater': '≫̸',
+ 'notgreaterless': '≹',
+ 'notgreaterslantequal': '⩾̸',
+ 'notgreatertilde': '≵',
+ 'nothumpdownhump': '≎̸',
+ 'nothumpequal': '≏̸',
+ 'notin': '∉',
+ 'notindot': '⋵̸',
+ 'notine': '⋹̸',
+ 'notinva': '∉',
+ 'notinvb': '⋷',
+ 'notinvc': '⋶',
+ 'notlefttriangle': '⋪',
+ 'notlefttrianglebar': '⧏̸',
+ 'notlefttriangleequal': '⋬',
+ 'notless': '≮',
+ 'notlessequal': '≰',
+ 'notlessgreater': '≸',
+ 'notlessless': '≪̸',
+ 'notlessslantequal': '⩽̸',
+ 'notlesstilde': '≴',
+ 'notnestedgreatergreater': '⪢̸',
+ 'notnestedlessless': '⪡̸',
+ 'notni': '∌',
+ 'notniva': '∌',
+ 'notnivb': '⋾',
+ 'notnivc': '⋽',
+ 'notprecedes': '⊀',
+ 'notprecedesequal': '⪯̸',
+ 'notprecedesslantequal': '⋠',
+ 'notreverseelement': '∌',
+ 'notrighttriangle': '⋫',
+ 'notrighttrianglebar': '⧐̸',
+ 'notrighttriangleequal': '⋭',
+ 'notsquaresubset': '⊏̸',
+ 'notsquaresubsetequal': '⋢',
+ 'notsquaresuperset': '⊐̸',
+ 'notsquaresupersetequal': '⋣',
+ 'notsubset': '⊂⃒',
+ 'notsubsetequal': '⊈',
+ 'notsucceeds': '⊁',
+ 'notsucceedsequal': '⪰̸',
+ 'notsucceedsslantequal': '⋡',
+ 'notsucceedstilde': '≿̸',
+ 'notsuperset': '⊃⃒',
+ 'notsupersetequal': '⊉',
+ 'nottilde': '≁',
+ 'nottildeequal': '≄',
+ 'nottildefullequal': '≇',
+ 'nottildetilde': '≉',
+ 'notverticalbar': '∤',
+ 'npar': '∦',
+ 'nparallel': '∦',
+ 'nparsl': '⫽⃥',
+ 'npart': '∂̸',
+ 'npolint': '⨔',
+ 'npr': '⊀',
+ 'nprcue': '⋠',
+ 'npre': '⪯̸',
+ 'nprec': '⊀',
+ 'npreceq': '⪯̸',
+ 'nrarr': '↛',
+ 'nrarrc': '⤳̸',
+ 'nrarrw': '↝̸',
+ 'nrightarrow': '↛',
+ 'nrtri': '⋫',
+ 'nrtrie': '⋭',
+ 'nsc': '⊁',
+ 'nsccue': '⋡',
+ 'nsce': '⪰̸',
+ 'nscr': '𝓃',
+ 'nshortmid': '∤',
+ 'nshortparallel': '∦',
+ 'nsim': '≁',
+ 'nsime': '≄',
+ 'nsimeq': '≄',
+ 'nsmid': '∤',
+ 'nspar': '∦',
+ 'nsqsube': '⋢',
+ 'nsqsupe': '⋣',
+ 'nsub': '⊄',
+ 'nsube': '⊈',
+ 'nsubset': '⊂⃒',
+ 'nsubseteq': '⊈',
+ 'nsubseteqq': '⫅̸',
+ 'nsucc': '⊁',
+ 'nsucceq': '⪰̸',
+ 'nsup': '⊅',
+ 'nsupe': '⊉',
+ 'nsupset': '⊃⃒',
+ 'nsupseteq': '⊉',
+ 'nsupseteqq': '⫆̸',
+ 'ntgl': '≹',
+ 'ntilde': 'ñ',
+ 'ntlg': '≸',
+ 'ntriangleleft': '⋪',
+ 'ntrianglelefteq': '⋬',
+ 'ntriangleright': '⋫',
+ 'ntrianglerighteq': '⋭',
+ 'nu': 'ν',
+ 'num': '#',
+ 'numero': '№',
+ 'numsp': '\u2007',
+ 'nvap': '≍⃒',
+ 'nvdash': '⊬',
+ 'nvge': '≥⃒',
+ 'nvgt': '>⃒',
+ 'nvharr': '⤄',
+ 'nvinfin': '⧞',
+ 'nvlarr': '⤂',
+ 'nvle': '≤⃒',
+ 'nvlt': '<⃒',
+ 'nvltrie': '⊴⃒',
+ 'nvrarr': '⤃',
+ 'nvrtrie': '⊵⃒',
+ 'nvsim': '∼⃒',
+ 'nwarhk': '⤣',
+ 'nwarr': '↖',
+ 'nwarrow': '↖',
+ 'nwnear': '⤧',
+ 'oacute': 'ó',
+ 'oast': '⊛',
+ 'ocir': '⊚',
+ 'ocirc': 'ô',
+ 'ocy': 'о',
+ 'odash': '⊝',
+ 'odblac': 'ő',
+ 'odiv': '⨸',
+ 'odot': '⊙',
+ 'odsold': '⦼',
+ 'oelig': 'œ',
+ 'ofcir': '⦿',
+ 'ofr': '𝔬',
+ 'ogon': '˛',
+ 'ograve': 'ò',
+ 'ogt': '⧁',
+ 'ohbar': '⦵',
+ 'ohm': 'Ω',
+ 'oint': '∮',
+ 'olarr': '↺',
+ 'olcir': '⦾',
+ 'olcross': '⦻',
+ 'oline': '‾',
+ 'olt': '⧀',
+ 'omacr': 'ō',
+ 'omega': 'ω',
+ 'omicron': 'ο',
+ 'omid': '⦶',
+ 'ominus': '⊖',
+ 'oopf': '𝕠',
+ 'opar': '⦷',
+ 'opencurlydoublequote': '“',
+ 'opencurlyquote': '‘',
+ 'operp': '⦹',
+ 'oplus': '⊕',
+ 'or': '∨',
+ 'orarr': '↻',
+ 'ord': '⩝',
+ 'order': 'ℴ',
+ 'orderof': 'ℴ',
+ 'ordf': 'ª',
+ 'ordm': 'º',
+ 'origof': '⊶',
+ 'oror': '⩖',
+ 'orslope': '⩗',
+ 'orv': '⩛',
+ 'os': 'Ⓢ',
+ 'oscr': 'ℴ',
+ 'oslash': 'ø',
+ 'osol': '⊘',
+ 'otilde': 'õ',
+ 'otimes': '⊗',
+ 'otimesas': '⨶',
+ 'ouml': 'ö',
+ 'ovbar': '⌽',
+ 'overbar': '‾',
+ 'overbrace': '⏞',
+ 'overbracket': '⎴',
+ 'overparenthesis': '⏜',
+ 'par': '∥',
+ 'para': '¶',
+ 'parallel': '∥',
+ 'parsim': '⫳',
+ 'parsl': '⫽',
+ 'part': '∂',
+ 'partiald': '∂',
+ 'pcy': 'п',
+ 'percnt': '%',
+ 'period': '.',
+ 'permil': '‰',
+ 'perp': '⊥',
+ 'pertenk': '‱',
+ 'pfr': '𝔭',
+ 'phi': 'φ',
+ 'phiv': 'ϕ',
+ 'phmmat': 'ℳ',
+ 'phone': '☎',
+ 'pi': 'π',
+ 'pitchfork': '⋔',
+ 'piv': 'ϖ',
+ 'planck': 'ℏ',
+ 'planckh': 'ℎ',
+ 'plankv': 'ℏ',
+ 'plus': '+',
+ 'plusacir': '⨣',
+ 'plusb': '⊞',
+ 'pluscir': '⨢',
+ 'plusdo': '∔',
+ 'plusdu': '⨥',
+ 'pluse': '⩲',
+ 'plusminus': '±',
+ 'plusmn': '±',
+ 'plussim': '⨦',
+ 'plustwo': '⨧',
+ 'pm': '±',
+ 'poincareplane': 'ℌ',
+ 'pointint': '⨕',
+ 'popf': '𝕡',
+ 'pound': '£',
+ 'pr': '≺',
+ 'prap': '⪷',
+ 'prcue': '≼',
+ 'pre': '⪯',
+ 'prec': '≺',
+ 'precapprox': '⪷',
+ 'preccurlyeq': '≼',
+ 'precedes': '≺',
+ 'precedesequal': '⪯',
+ 'precedesslantequal': '≼',
+ 'precedestilde': '≾',
+ 'preceq': '⪯',
+ 'precnapprox': '⪹',
+ 'precneqq': '⪵',
+ 'precnsim': '⋨',
+ 'precsim': '≾',
+ 'prime': '′',
+ 'primes': 'ℙ',
+ 'prnap': '⪹',
+ 'prne': '⪵',
+ 'prnsim': '⋨',
+ 'prod': '∏',
+ 'product': '∏',
+ 'profalar': '⌮',
+ 'profline': '⌒',
+ 'profsurf': '⌓',
+ 'prop': '∝',
+ 'proportion': '∷',
+ 'proportional': '∝',
+ 'propto': '∝',
+ 'prsim': '≾',
+ 'prurel': '⊰',
+ 'pscr': '𝓅',
+ 'psi': 'ψ',
+ 'puncsp': '\u2008',
+ 'qfr': '𝔮',
+ 'qint': '⨌',
+ 'qopf': '𝕢',
+ 'qprime': '⁗',
+ 'qscr': '𝓆',
+ 'quaternions': 'ℍ',
+ 'quatint': '⨖',
+ 'quest': '?',
+ 'questeq': '≟',
+ 'quot': '"',
+ 'raarr': '⇛',
+ 'race': '∽̱',
+ 'racute': 'ŕ',
+ 'radic': '√',
+ 'raemptyv': '⦳',
+ 'rang': '⟩',
+ 'rangd': '⦒',
+ 'range': '⦥',
+ 'rangle': '⟩',
+ 'raquo': '»',
+ 'rarr': '→',
+ 'rarrap': '⥵',
+ 'rarrb': '⇥',
+ 'rarrbfs': '⤠',
+ 'rarrc': '⤳',
+ 'rarrfs': '⤞',
+ 'rarrhk': '↪',
+ 'rarrlp': '↬',
+ 'rarrpl': '⥅',
+ 'rarrsim': '⥴',
+ 'rarrtl': '↣',
+ 'rarrw': '↝',
+ 'ratail': '⤚',
+ 'ratio': '∶',
+ 'rationals': 'ℚ',
+ 'rbarr': '⤍',
+ 'rbbrk': '❳',
+ 'rbrace': '}',
+ 'rbrack': ']',
+ 'rbrke': '⦌',
+ 'rbrksld': '⦎',
+ 'rbrkslu': '⦐',
+ 'rcaron': 'ř',
+ 'rcedil': 'ŗ',
+ 'rceil': '⌉',
+ 'rcub': '}',
+ 'rcy': 'р',
+ 'rdca': '⤷',
+ 'rdldhar': '⥩',
+ 'rdquo': '”',
+ 'rdquor': '”',
+ 'rdsh': '↳',
+ 're': 'ℜ',
+ 'real': 'ℜ',
+ 'realine': 'ℛ',
+ 'realpart': 'ℜ',
+ 'reals': 'ℝ',
+ 'rect': '▭',
+ 'reg': '®',
+ 'reverseelement': '∋',
+ 'reverseequilibrium': '⇋',
+ 'reverseupequilibrium': '⥯',
+ 'rfisht': '⥽',
+ 'rfloor': '⌋',
+ 'rfr': '𝔯',
+ 'rhar': '⥤',
+ 'rhard': '⇁',
+ 'rharu': '⇀',
+ 'rharul': '⥬',
+ 'rho': 'ρ',
+ 'rhov': 'ϱ',
+ 'rightanglebracket': '⟩',
+ 'rightarrow': '→',
+ 'rightarrowbar': '⇥',
+ 'rightarrowleftarrow': '⇄',
+ 'rightarrowtail': '↣',
+ 'rightceiling': '⌉',
+ 'rightdoublebracket': '⟧',
+ 'rightdownteevector': '⥝',
+ 'rightdownvector': '⇂',
+ 'rightdownvectorbar': '⥕',
+ 'rightfloor': '⌋',
+ 'rightharpoondown': '⇁',
+ 'rightharpoonup': '⇀',
+ 'rightleftarrows': '⇄',
+ 'rightleftharpoons': '⇌',
+ 'rightrightarrows': '⇉',
+ 'rightsquigarrow': '↝',
+ 'righttee': '⊢',
+ 'rightteearrow': '↦',
+ 'rightteevector': '⥛',
+ 'rightthreetimes': '⋌',
+ 'righttriangle': '⊳',
+ 'righttrianglebar': '⧐',
+ 'righttriangleequal': '⊵',
+ 'rightupdownvector': '⥏',
+ 'rightupteevector': '⥜',
+ 'rightupvector': '↾',
+ 'rightupvectorbar': '⥔',
+ 'rightvector': '⇀',
+ 'rightvectorbar': '⥓',
+ 'ring': '˚',
+ 'risingdotseq': '≓',
+ 'rlarr': '⇄',
+ 'rlhar': '⇌',
+ 'rlm': '\u200f',
+ 'rmoust': '⎱',
+ 'rmoustache': '⎱',
+ 'rnmid': '⫮',
+ 'roang': '⟭',
+ 'roarr': '⇾',
+ 'robrk': '⟧',
+ 'ropar': '⦆',
+ 'ropf': '𝕣',
+ 'roplus': '⨮',
+ 'rotimes': '⨵',
+ 'roundimplies': '⥰',
+ 'rpar': ')',
+ 'rpargt': '⦔',
+ 'rppolint': '⨒',
+ 'rrarr': '⇉',
+ 'rrightarrow': '⇛',
+ 'rsaquo': '›',
+ 'rscr': '𝓇',
+ 'rsh': '↱',
+ 'rsqb': ']',
+ 'rsquo': '’',
+ 'rsquor': '’',
+ 'rthree': '⋌',
+ 'rtimes': '⋊',
+ 'rtri': '▹',
+ 'rtrie': '⊵',
+ 'rtrif': '▸',
+ 'rtriltri': '⧎',
+ 'ruledelayed': '⧴',
+ 'ruluhar': '⥨',
+ 'rx': '℞',
+ 'sacute': 'ś',
+ 'sbquo': '‚',
+ 'sc': '≻',
+ 'scap': '⪸',
+ 'scaron': 'š',
+ 'sccue': '≽',
+ 'sce': '⪰',
+ 'scedil': 'ş',
+ 'scirc': 'ŝ',
+ 'scnap': '⪺',
+ 'scne': '⪶',
+ 'scnsim': '⋩',
+ 'scpolint': '⨓',
+ 'scsim': '≿',
+ 'scy': 'с',
+ 'sdot': '⋅',
+ 'sdotb': '⊡',
+ 'sdote': '⩦',
+ 'searhk': '⤥',
+ 'searr': '↘',
+ 'searrow': '↘',
+ 'sect': '§',
+ 'semi': ';',
+ 'seswar': '⤩',
+ 'setminus': '∖',
+ 'setmn': '∖',
+ 'sext': '✶',
+ 'sfr': '𝔰',
+ 'sfrown': '⌢',
+ 'sharp': '♯',
+ 'shchcy': 'щ',
+ 'shcy': 'ш',
+ 'shortdownarrow': '↓',
+ 'shortleftarrow': '←',
+ 'shortmid': '∣',
+ 'shortparallel': '∥',
+ 'shortrightarrow': '→',
+ 'shortuparrow': '↑',
+ 'shy': '\xad',
+ 'sigma': 'σ',
+ 'sigmaf': 'ς',
+ 'sigmav': 'ς',
+ 'sim': '∼',
+ 'simdot': '⩪',
+ 'sime': '≃',
+ 'simeq': '≃',
+ 'simg': '⪞',
+ 'simge': '⪠',
+ 'siml': '⪝',
+ 'simle': '⪟',
+ 'simne': '≆',
+ 'simplus': '⨤',
+ 'simrarr': '⥲',
+ 'slarr': '←',
+ 'smallcircle': '∘',
+ 'smallsetminus': '∖',
+ 'smashp': '⨳',
+ 'smeparsl': '⧤',
+ 'smid': '∣',
+ 'smile': '⌣',
+ 'smt': '⪪',
+ 'smte': '⪬',
+ 'smtes': '⪬︀',
+ 'softcy': 'ь',
+ 'sol': '/',
+ 'solb': '⧄',
+ 'solbar': '⌿',
+ 'sopf': '𝕤',
+ 'spades': '♠',
+ 'spadesuit': '♠',
+ 'spar': '∥',
+ 'sqcap': '⊓',
+ 'sqcaps': '⊓︀',
+ 'sqcup': '⊔',
+ 'sqcups': '⊔︀',
+ 'sqrt': '√',
+ 'sqsub': '⊏',
+ 'sqsube': '⊑',
+ 'sqsubset': '⊏',
+ 'sqsubseteq': '⊑',
+ 'sqsup': '⊐',
+ 'sqsupe': '⊒',
+ 'sqsupset': '⊐',
+ 'sqsupseteq': '⊒',
+ 'squ': '□',
+ 'square': '□',
+ 'squareintersection': '⊓',
+ 'squaresubset': '⊏',
+ 'squaresubsetequal': '⊑',
+ 'squaresuperset': '⊐',
+ 'squaresupersetequal': '⊒',
+ 'squareunion': '⊔',
+ 'squarf': '▪',
+ 'squf': '▪',
+ 'srarr': '→',
+ 'sscr': '𝓈',
+ 'ssetmn': '∖',
+ 'ssmile': '⌣',
+ 'sstarf': '⋆',
+ 'star': '☆',
+ 'starf': '★',
+ 'straightepsilon': 'ϵ',
+ 'straightphi': 'ϕ',
+ 'strns': '¯',
+ 'sub': '⊂',
+ 'subdot': '⪽',
+ 'sube': '⊆',
+ 'subedot': '⫃',
+ 'submult': '⫁',
+ 'subne': '⊊',
+ 'subplus': '⪿',
+ 'subrarr': '⥹',
+ 'subset': '⊂',
+ 'subseteq': '⊆',
+ 'subseteqq': '⫅',
+ 'subsetequal': '⊆',
+ 'subsetneq': '⊊',
+ 'subsetneqq': '⫋',
+ 'subsim': '⫇',
+ 'subsub': '⫕',
+ 'subsup': '⫓',
+ 'succ': '≻',
+ 'succapprox': '⪸',
+ 'succcurlyeq': '≽',
+ 'succeeds': '≻',
+ 'succeedsequal': '⪰',
+ 'succeedsslantequal': '≽',
+ 'succeedstilde': '≿',
+ 'succeq': '⪰',
+ 'succnapprox': '⪺',
+ 'succneqq': '⪶',
+ 'succnsim': '⋩',
+ 'succsim': '≿',
+ 'suchthat': '∋',
+ 'sum': '∑',
+ 'sung': '♪',
+ 'sup': '⊃',
+ 'sup1': '¹',
+ 'sup2': '²',
+ 'sup3': '³',
+ 'supdot': '⪾',
+ 'supdsub': '⫘',
+ 'supe': '⊇',
+ 'supedot': '⫄',
+ 'superset': '⊃',
+ 'supersetequal': '⊇',
+ 'suphsol': '⟉',
+ 'suphsub': '⫗',
+ 'suplarr': '⥻',
+ 'supmult': '⫂',
+ 'supne': '⊋',
+ 'supplus': '⫀',
+ 'supset': '⊃',
+ 'supseteq': '⊇',
+ 'supseteqq': '⫆',
+ 'supsetneq': '⊋',
+ 'supsetneqq': '⫌',
+ 'supsim': '⫈',
+ 'supsub': '⫔',
+ 'supsup': '⫖',
+ 'swarhk': '⤦',
+ 'swarr': '↙',
+ 'swarrow': '↙',
+ 'swnwar': '⤪',
+ 'szlig': 'ß',
+ 'tab': ' ',
+ 'target': '⌖',
+ 'tau': 'τ',
+ 'tbrk': '⎴',
+ 'tcaron': 'ť',
+ 'tcedil': 'ţ',
+ 'tcy': 'т',
+ 'tdot': '⃛',
+ 'telrec': '⌕',
+ 'tfr': '𝔱',
+ 'there4': '∴',
+ 'therefore': '∴',
+ 'theta': 'θ',
+ 'thetasym': 'ϑ',
+ 'thetav': 'ϑ',
+ 'thickapprox': '≈',
+ 'thicksim': '∼',
+ 'thickspace': '\u205f\u200a',
+ 'thinsp': '\u2009',
+ 'thinspace': '\u2009',
+ 'thkap': '≈',
+ 'thksim': '∼',
+ 'thorn': 'þ',
+ 'tilde': '˜',
+ 'tildeequal': '≃',
+ 'tildefullequal': '≅',
+ 'tildetilde': '≈',
+ 'times': '×',
+ 'timesb': '⊠',
+ 'timesbar': '⨱',
+ 'timesd': '⨰',
+ 'tint': '∭',
+ 'toea': '⤨',
+ 'top': '⊤',
+ 'topbot': '⌶',
+ 'topcir': '⫱',
+ 'topf': '𝕥',
+ 'topfork': '⫚',
+ 'tosa': '⤩',
+ 'tprime': '‴',
+ 'trade': '™',
+ 'triangle': '▵',
+ 'triangledown': '▿',
+ 'triangleleft': '◃',
+ 'trianglelefteq': '⊴',
+ 'triangleq': '≜',
+ 'triangleright': '▹',
+ 'trianglerighteq': '⊵',
+ 'tridot': '◬',
+ 'trie': '≜',
+ 'triminus': '⨺',
+ 'tripledot': '⃛',
+ 'triplus': '⨹',
+ 'trisb': '⧍',
+ 'tritime': '⨻',
+ 'trpezium': '⏢',
+ 'tscr': '𝓉',
+ 'tscy': 'ц',
+ 'tshcy': 'ћ',
+ 'tstrok': 'ŧ',
+ 'twixt': '≬',
+ 'twoheadleftarrow': '↞',
+ 'twoheadrightarrow': '↠',
+ 'uacute': 'ú',
+ 'uarr': '↑',
+ 'uarrocir': '⥉',
+ 'ubrcy': 'ў',
+ 'ubreve': 'ŭ',
+ 'ucirc': 'û',
+ 'ucy': 'у',
+ 'udarr': '⇅',
+ 'udblac': 'ű',
+ 'udhar': '⥮',
+ 'ufisht': '⥾',
+ 'ufr': '𝔲',
+ 'ugrave': 'ù',
+ 'uhar': '⥣',
+ 'uharl': '↿',
+ 'uharr': '↾',
+ 'uhblk': '▀',
+ 'ulcorn': '⌜',
+ 'ulcorner': '⌜',
+ 'ulcrop': '⌏',
+ 'ultri': '◸',
+ 'umacr': 'ū',
+ 'uml': '¨',
+ 'underbar': '_',
+ 'underbrace': '⏟',
+ 'underbracket': '⎵',
+ 'underparenthesis': '⏝',
+ 'union': '⋃',
+ 'unionplus': '⊎',
+ 'uogon': 'ų',
+ 'uopf': '𝕦',
+ 'uparrow': '↑',
+ 'uparrowbar': '⤒',
+ 'uparrowdownarrow': '⇅',
+ 'updownarrow': '↕',
+ 'upequilibrium': '⥮',
+ 'upharpoonleft': '↿',
+ 'upharpoonright': '↾',
+ 'uplus': '⊎',
+ 'upperleftarrow': '↖',
+ 'upperrightarrow': '↗',
+ 'upsi': 'υ',
+ 'upsih': 'ϒ',
+ 'upsilon': 'υ',
+ 'uptee': '⊥',
+ 'upteearrow': '↥',
+ 'upuparrows': '⇈',
+ 'urcorn': '⌝',
+ 'urcorner': '⌝',
+ 'urcrop': '⌎',
+ 'uring': 'ů',
+ 'urtri': '◹',
+ 'uscr': '𝓊',
+ 'utdot': '⋰',
+ 'utilde': 'ũ',
+ 'utri': '▵',
+ 'utrif': '▴',
+ 'uuarr': '⇈',
+ 'uuml': 'ü',
+ 'uwangle': '⦧',
+ 'vangrt': '⦜',
+ 'varepsilon': 'ϵ',
+ 'varkappa': 'ϰ',
+ 'varnothing': '∅',
+ 'varphi': 'ϕ',
+ 'varpi': 'ϖ',
+ 'varpropto': '∝',
+ 'varr': '↕',
+ 'varrho': 'ϱ',
+ 'varsigma': 'ς',
+ 'varsubsetneq': '⊊︀',
+ 'varsubsetneqq': '⫋︀',
+ 'varsupsetneq': '⊋︀',
+ 'varsupsetneqq': '⫌︀',
+ 'vartheta': 'ϑ',
+ 'vartriangleleft': '⊲',
+ 'vartriangleright': '⊳',
+ 'vbar': '⫨',
+ 'vbarv': '⫩',
+ 'vcy': 'в',
+ 'vdash': '⊢',
+ 'vdashl': '⫦',
+ 'vee': '∨',
+ 'veebar': '⊻',
+ 'veeeq': '≚',
+ 'vellip': '⋮',
+ 'verbar': '|',
+ 'vert': '|',
+ 'verticalbar': '∣',
+ 'verticalline': '|',
+ 'verticalseparator': '❘',
+ 'verticaltilde': '≀',
+ 'verythinspace': '\u200a',
+ 'vfr': '𝔳',
+ 'vltri': '⊲',
+ 'vnsub': '⊂⃒',
+ 'vnsup': '⊃⃒',
+ 'vopf': '𝕧',
+ 'vprop': '∝',
+ 'vrtri': '⊳',
+ 'vscr': '𝓋',
+ 'vsubne': '⊊︀',
+ 'vsupne': '⊋︀',
+ 'vvdash': '⊪',
+ 'vzigzag': '⦚',
+ 'wcirc': 'ŵ',
+ 'wedbar': '⩟',
+ 'wedge': '∧',
+ 'wedgeq': '≙',
+ 'weierp': '℘',
+ 'wfr': '𝔴',
+ 'wopf': '𝕨',
+ 'wp': '℘',
+ 'wr': '≀',
+ 'wreath': '≀',
+ 'wscr': '𝓌',
+ 'xcap': '⋂',
+ 'xcirc': '◯',
+ 'xcup': '⋃',
+ 'xdtri': '▽',
+ 'xfr': '𝔵',
+ 'xharr': '⟷',
+ 'xi': 'ξ',
+ 'xlarr': '⟵',
+ 'xmap': '⟼',
+ 'xnis': '⋻',
+ 'xodot': '⨀',
+ 'xopf': '𝕩',
+ 'xoplus': '⨁',
+ 'xotime': '⨂',
+ 'xrarr': '⟶',
+ 'xscr': '𝓍',
+ 'xsqcup': '⨆',
+ 'xuplus': '⨄',
+ 'xutri': '△',
+ 'xvee': '⋁',
+ 'xwedge': '⋀',
+ 'yacute': 'ý',
+ 'yacy': 'я',
+ 'ycirc': 'ŷ',
+ 'ycy': 'ы',
+ 'yen': '¥',
+ 'yfr': '𝔶',
+ 'yicy': 'ї',
+ 'yopf': '𝕪',
+ 'yscr': '𝓎',
+ 'yucy': 'ю',
+ 'yuml': 'ÿ',
+ 'zacute': 'ź',
+ 'zcaron': 'ž',
+ 'zcy': 'з',
+ 'zdot': 'ż',
+ 'zeetrf': 'ℨ',
+ 'zerowidthspace': '\u200b',
+ 'zeta': 'ζ',
+ 'zfr': '𝔷',
+ 'zhcy': 'ж',
+ 'zigrarr': '⇝',
+ 'zopf': '𝕫',
+ 'zscr': '𝓏',
+ 'zwj': '\u200d',
+ 'zwnj': '\u200c',
+# }}}
+}
+
+
+if __name__ == '__main__':
+ import re
+ from html5lib.constants import entities
+ entities = {k.lower().replace(';', ''): entities[k] for k in entities}
+ lines = []
+
+ for k in sorted(entities):
+ lines.append(" '%s': %r," % (k, entities[k]))
+
+ with open(__file__, 'r+b') as f:
+ raw = f.read().decode('utf-8')
+ raw = re.sub(r'^# ENTITY_DATA {{{.+^# }}}',
+ '# ENTITY_DATA {{{\n' + '\n'.join(lines) + '\n# }}}',
+ raw, flags=re.M | re.DOTALL)
+ f.seek(0), f.truncate()
+ f.write(raw.encode('utf-8'))
diff --git a/core/lib/mechanize/_equiv.py b/core/lib/mechanize/_equiv.py
new file mode 100644
index 00000000..6c00287a
--- /dev/null
+++ b/core/lib/mechanize/_equiv.py
@@ -0,0 +1,350 @@
+#!/usr/bin/env python
+# vim:fileencoding=utf-8
+# Copyright: 2017, Kovid Goyal
+
+from __future__ import (absolute_import, division, print_function,
+ unicode_literals)
+
+import re
+import string
+
+from ._entities import html5_entities
+from .polyglot import codepoint_to_chr
+
+space_chars = frozenset(("\t", "\n", "\u000C", " ", "\r"))
+space_chars_bytes = frozenset(item.encode("ascii") for item in space_chars)
+ascii_letters_bytes = frozenset(
+ item.encode("ascii") for item in string.ascii_letters)
+spaces_angle_brackets = space_chars_bytes | frozenset((b">", b"<"))
+skip1 = space_chars_bytes | frozenset((b"/", ))
+head_elems = frozenset((
+ b"html", b"head", b"title", b"base", b"script",
+ b"style", b"meta", b"link", b"object"))
+
+
+def my_unichr(num):
+ try:
+ return codepoint_to_chr(num)
+ except (ValueError, OverflowError):
+ return '?'
+
+
+def replace_entity(match):
+ ent = match.group(1).lower()
+ if ent in {'apos', 'squot'}:
+ # squot is generated by some broken CMS software
+ return "'"
+ if ent == 'hellips':
+ ent = 'hellip'
+ if ent.startswith('#'):
+ try:
+ if ent[1] in ('x', 'X'):
+ num = int(ent[2:], 16)
+ else:
+ num = int(ent[1:])
+ except Exception:
+ return '&' + ent + ';'
+ if num > 255:
+ return my_unichr(num)
+ try:
+ return chr(num).decode('cp1252')
+ except UnicodeDecodeError:
+ return my_unichr(num)
+ try:
+ return html5_entities[ent]
+ except KeyError:
+ pass
+ return '&' + ent + ';'
+
+
+class Bytes(bytes):
+ """String-like object with an associated position and various extra methods
+ If the position is ever greater than the string length then an exception is
+ raised"""
+
+ def __init__(self, value):
+ self._position = -1
+
+ def __iter__(self):
+ return self
+
+ def __next__(self):
+ p = self._position = self._position + 1
+ if p >= len(self):
+ raise StopIteration
+ elif p < 0:
+ raise TypeError
+ return self[p:p + 1]
+
+ def next(self):
+ # Py2 compat
+ return self.__next__()
+
+ def previous(self):
+ p = self._position
+ if p >= len(self):
+ raise StopIteration
+ elif p < 0:
+ raise TypeError
+ self._position = p = p - 1
+ return self[p:p + 1]
+
+ @property
+ def position(self):
+ if self._position >= len(self):
+ raise StopIteration
+ if self._position >= 0:
+ return self._position
+
+ @position.setter
+ def position(self, position):
+ if self._position >= len(self):
+ raise StopIteration
+ self._position = position
+
+ @property
+ def current_byte(self):
+ return self[self.position:self.position + 1]
+
+ def skip(self, chars=space_chars_bytes):
+ """Skip past a list of characters"""
+ p = self.position # use property for the error-checking
+ while p < len(self):
+ c = self[p:p + 1]
+ if c not in chars:
+ self._position = p
+ return c
+ p += 1
+ self._position = p
+ return
+
+ def skip_until(self, chars):
+ p = pos = self.position
+ while p < len(self):
+ c = self[p:p + 1]
+ if c in chars:
+ self._position = p
+ return self[pos:p], c
+ p += 1
+ self._position = p
+ return b'', b''
+
+ def match_bytes(self, bytes):
+ """Look for a sequence of bytes at the start of a string. If the bytes
+ are found return True and advance the position to the byte after the
+ match. Otherwise return False and leave the position alone"""
+ p = self.position
+ data = self[p:p + len(bytes)]
+ rv = data.startswith(bytes)
+ if rv:
+ self.position += len(bytes)
+ return rv
+
+ def match_bytes_pat(self, pat):
+ bytes = pat.pattern
+ m = pat.match(self, self.position)
+ if m is None:
+ return False
+ bytes = m.group()
+ self.position += len(bytes)
+ return True
+
+ def jump_to(self, bytes):
+ """Look for the next sequence of bytes matching a given sequence. If
+ a match is found advance the position to the last byte of the match"""
+ new_pos = self.find(bytes, max(0, self.position))
+ if new_pos > -1:
+ new_pos -= self.position
+ if self._position == -1:
+ self._position = 0
+ self._position += (new_pos + len(bytes) - 1)
+ return True
+ else:
+ raise StopIteration
+
+
+class HTTPEquivParser(object):
+ """Mini parser for detecting http-equiv headers from meta tags """
+
+ def __init__(self, data):
+ """string - the data to work on """
+ self.data = Bytes(data)
+ self.headers = []
+
+ def __call__(self):
+ mb, mbp = self.data.match_bytes, self.data.match_bytes_pat
+ dispatch = (
+ (mb, b"")
+
+ def handle_meta(self):
+ if self.data.current_byte not in space_chars_bytes:
+ # if we have ' to addheader(),
+ start*body() affects where the header is inserted; 0 means
+ append at the end, 1 means insert at the start; default is
+ append for addheader(), but insert for start*body(), which use
+ it to determine where the Content-type header goes.
+
+ """
+
+ def __init__(self, fp, http_hdrs=None):
+ self._http_hdrs = http_hdrs
+ self._fp = fp
+ self._headers = []
+ self._boundary = []
+ self._first_part = True
+
+ def addheader(self, key, value, prefix=0, add_to_http_hdrs=0):
+ """
+ prefix is ignored if add_to_http_hdrs is true.
+ """
+ lines = value.split("\r\n")
+ while lines and not lines[-1]:
+ del lines[-1]
+ while lines and not lines[0]:
+ del lines[0]
+ if add_to_http_hdrs:
+ value = "".join(lines)
+ # 2.2 urllib2 doesn't normalize header case
+ self._http_hdrs.append((key.capitalize(), value))
+ else:
+ for i in range(1, len(lines)):
+ lines[i] = " " + lines[i].strip()
+ value = "\r\n".join(lines) + "\r\n"
+ line = key.title() + ": " + value
+ if prefix:
+ self._headers.insert(0, line)
+ else:
+ self._headers.append(line)
+
+ def flushheaders(self):
+ for line in self._headers:
+ self._fp.write(line)
+ self._headers = []
+
+ def startbody(self,
+ ctype=None,
+ plist=[],
+ prefix=1,
+ add_to_http_hdrs=0,
+ content_type=1):
+ """
+ prefix is ignored if add_to_http_hdrs is true.
+ """
+ if content_type and ctype:
+ for name, value in plist:
+ ctype = ctype + ';\r\n %s=%s' % (name, value)
+ self.addheader(
+ "Content-Type",
+ ctype,
+ prefix=prefix,
+ add_to_http_hdrs=add_to_http_hdrs)
+ self.flushheaders()
+ if not add_to_http_hdrs:
+ self._fp.write("\r\n")
+ self._first_part = True
+ return self._fp
+
+ def startmultipartbody(self,
+ subtype,
+ boundary=None,
+ plist=[],
+ prefix=1,
+ add_to_http_hdrs=0,
+ content_type=1):
+ boundary = boundary or choose_boundary()
+ self._boundary.append(boundary)
+ return self.startbody(
+ "multipart/" + subtype, [("boundary", boundary)] + plist,
+ prefix=prefix,
+ add_to_http_hdrs=add_to_http_hdrs,
+ content_type=content_type)
+
+ def nextpart(self):
+ boundary = self._boundary[-1]
+ if self._first_part:
+ self._first_part = False
+ else:
+ self._fp.write("\r\n")
+ self._fp.write("--" + boundary + "\r\n")
+ return self.__class__(self._fp)
+
+ def lastpart(self):
+ if self._first_part:
+ self.nextpart()
+ boundary = self._boundary.pop()
+ self._fp.write("\r\n--" + boundary + "--\r\n")
+
+
+class Label:
+ def __init__(self, text, for_id=None):
+ self.id = for_id
+ self.text = compress_whitespace(text or '')
+
+ def __repr__(self):
+ return "" % (self.id, self.text)
+
+ __str__ = __repr__
+
+
+def _get_label(attrs):
+ text = attrs.get("__label")
+ if text is not None:
+ return Label(text)
+
+
+class Control:
+ """An HTML form control.
+
+ An HTMLForm contains a sequence of Controls. The Controls in an HTMLForm
+ are accessed using the HTMLForm.find_control method or the
+ HTMLForm.controls attribute.
+
+ Control instances are usually constructed using the ParseFile /
+ ParseResponse functions. If you use those functions, you can ignore the
+ rest of this paragraph. A Control is only properly initialised after the
+ fixup method has been called. In fact, this is only strictly necessary for
+ ListControl instances. This is necessary because ListControls are built up
+ from ListControls each containing only a single item, and their initial
+ value(s) can only be known after the sequence is complete.
+
+ The types and values that are acceptable for assignment to the value
+ attribute are defined by subclasses.
+
+ If the disabled attribute is true, this represents the state typically
+ represented by browsers by 'greying out' a control. If the disabled
+ attribute is true, the Control will raise AttributeError if an attempt is
+ made to change its value. In addition, the control will not be considered
+ 'successful' as defined by the W3C HTML 4 standard -- ie. it will
+ contribute no data to the return value of the HTMLForm.click* methods. To
+ enable a control, set the disabled attribute to a false value.
+
+ If the readonly attribute is true, the Control will raise AttributeError if
+ an attempt is made to change its value. To make a control writable, set
+ the readonly attribute to a false value.
+
+ All controls have the disabled and readonly attributes, not only those that
+ may have the HTML attributes of the same names.
+
+ On assignment to the value attribute, the following exceptions are raised:
+ TypeError, AttributeError (if the value attribute should not be assigned
+ to, because the control is disabled, for example) and ValueError.
+
+ If the name or value attributes are None, or the value is an empty list, or
+ if the control is disabled, the control is not successful.
+
+ Public attributes:
+
+ :ivar str type: string describing type of control (see the keys of the
+ HTMLForm.type2class dictionary for the allowable values) (readonly)
+ :ivar str name: name of control (readonly)
+ :ivar value: current value of control (subclasses may allow a single value,
+ a sequence of values, or either)
+ :ivar bool disabled: disabled state
+ :ivar bool readonly: readonly state
+ :ivar str id: value of id HTML attribute
+
+ """
+
+ def __init__(self, type, name, attrs, index=None):
+ """
+ type: string describing type of control (see the keys of the
+ HTMLForm.type2class dictionary for the allowable values)
+ name: control name
+ attrs: HTML attributes of control's HTML element
+
+ """
+ raise NotImplementedError()
+
+ def add_to_form(self, form):
+ self._form = form
+ form.controls.append(self)
+
+ def fixup(self):
+ pass
+
+ def is_of_kind(self, kind):
+ raise NotImplementedError()
+
+ def clear(self):
+ raise NotImplementedError()
+
+ def __getattr__(self, name):
+ raise NotImplementedError()
+
+ def __setattr__(self, name, value):
+ raise NotImplementedError()
+
+ def pairs(self):
+ """Return list of (key, value) pairs suitable for passing to urlencode.
+ """
+ return [(k, v) for (i, k, v) in self._totally_ordered_pairs()]
+
+ def _totally_ordered_pairs(self):
+ """Return list of (key, value, index) tuples.
+
+ Like pairs, but allows preserving correct ordering even where several
+ controls are involved.
+
+ """
+ raise NotImplementedError()
+
+ def _write_mime_data(self, mw, name, value):
+ """Write data for a subitem of this control to a MimeWriter."""
+ # called by HTMLForm
+ mw2 = mw.nextpart()
+ mw2.addheader(
+ "Content-Disposition", 'form-data; name="%s"' % as_unicode(name),
+ 1)
+ f = mw2.startbody(prefix=0)
+ f.write(value)
+
+ def __str__(self):
+ raise NotImplementedError()
+
+ def get_labels(self):
+ """Return all labels (Label instances) for this control.
+
+ If the control was surrounded by a tag, that will be the first
+ label; all other labels, connected by 'for' and 'id', are in the order
+ that appear in the HTML.
+
+ """
+ res = []
+ if self._label:
+ res.append(self._label)
+ if self.id:
+ res.extend(self._form._id_to_labels.get(self.id, ()))
+ return res
+
+
+# ---------------------------------------------------
+class ScalarControl(Control):
+ """Control whose value is not restricted to one of a prescribed set.
+
+ Some ScalarControls don't accept any value attribute. Otherwise, takes a
+ single value, which must be string-like.
+
+ Additional read-only public attribute:
+
+ :ivar dict attrs: dictionary mapping the names of original HTML attributes
+ of the control to their values
+
+ """
+
+ def __init__(self, type, name, attrs, index=None):
+ self._index = index
+ self._label = _get_label(attrs)
+ self.__dict__["type"] = type.lower()
+ self.__dict__["name"] = name
+ self._value = attrs.get("value")
+ self.disabled = 'disabled' in attrs
+ self.readonly = 'readonly' in attrs
+ self.id = attrs.get("id")
+
+ self.attrs = dict(attrs)
+
+ self._clicked = False
+
+ self._urlparse = urlparse
+ self._urlunparse = urlunparse
+
+ def __getattr__(self, name):
+ if name == "value":
+ return self.__dict__["_value"]
+ else:
+ raise AttributeError("%s instance has no attribute '%s'" %
+ (self.__class__.__name__, name))
+
+ def __setattr__(self, name, value):
+ if name == "value":
+ if not isstringlike(value):
+ raise TypeError("must assign a string")
+ elif self.readonly:
+ raise AttributeError("control '%s' is readonly" % self.name)
+ elif self.disabled:
+ raise AttributeError("control '%s' is disabled" % self.name)
+ self.__dict__["_value"] = value
+ elif name in ("name", "type"):
+ raise AttributeError("%s attribute is readonly" % name)
+ else:
+ self.__dict__[name] = value
+
+ def _totally_ordered_pairs(self):
+ name = self.name
+ value = self.value
+ if name is None or value is None or self.disabled:
+ return []
+ return [(self._index, name, value)]
+
+ def clear(self):
+ if self.readonly:
+ raise AttributeError("control '%s' is readonly" % self.name)
+ self.__dict__["_value"] = None
+
+ def __str__(self):
+ name = self.name
+ value = self.value
+ if name is None:
+ name = ""
+ if value is None:
+ value = ""
+
+ infos = []
+ if self.disabled:
+ infos.append("disabled")
+ if self.readonly:
+ infos.append("readonly")
+ info = ", ".join(infos)
+ if info:
+ info = " (%s)" % info
+
+ return "<%s(%s=%s)%s>" % (self.__class__.__name__, name, value, info)
+
+
+# ---------------------------------------------------
+class TextControl(ScalarControl):
+ """Textual input control.
+
+ Covers HTML elements: INPUT/TEXT, INPUT/PASSWORD, INPUT/HIDDEN, TEXTAREA
+
+ """
+
+ def __init__(self, type, name, attrs, index=None):
+ ScalarControl.__init__(self, type, name, attrs, index)
+ if self.type == "hidden":
+ self.readonly = True
+ if self._value is None:
+ self._value = ""
+
+ def is_of_kind(self, kind):
+ return kind == "text"
+
+
+# ---------------------------------------------------
+class FileControl(ScalarControl):
+ """File upload with INPUT TYPE=FILE.
+
+ The value attribute of a FileControl is always None. Use add_file instead.
+
+ Additional public method: :meth:`add_file`
+
+ """
+
+ def __init__(self, type, name, attrs, index=None):
+ ScalarControl.__init__(self, type, name, attrs, index)
+ self._value = None
+ self._upload_data = []
+
+ def is_of_kind(self, kind):
+ return kind == "file"
+
+ def clear(self):
+ if self.readonly:
+ raise AttributeError("control '%s' is readonly" % self.name)
+ self._upload_data = []
+
+ def __setattr__(self, name, value):
+ if name in ("value", "name", "type"):
+ raise AttributeError("%s attribute is readonly" % name)
+ else:
+ self.__dict__[name] = value
+
+ def add_file(self, file_object, content_type=None, filename=None):
+ ''' Add data from the specified file to be uploaded. content_type and
+ filename are sent in the HTTP headers if specified. '''
+ if not hasattr(file_object, "read"):
+ raise TypeError("file-like object must have read method")
+ if content_type is not None and not isstringlike(content_type):
+ raise TypeError("content type must be None or string-like")
+ if filename is not None and not isstringlike(filename):
+ raise TypeError("filename must be None or string-like")
+ if content_type is None:
+ if getattr(file_object, 'name', None):
+ content_type = guess_type(file_object.name)[0]
+ content_type = content_type or "application/octet-stream"
+ self._upload_data.append((file_object, content_type, filename))
+
+ def _totally_ordered_pairs(self):
+ # XXX should it be successful even if unnamed?
+ if self.name is None or self.disabled:
+ return []
+ return [(self._index, self.name, "")]
+
+ # If enctype is application/x-www-form-urlencoded and there's a FILE
+ # control present, what should be sent? Strictly, it should be 'name=data'
+ # (see HTML 4.01 spec., section 17.13.2), but code sends "name=" ATM. What
+ # about multiple file upload?
+ def _write_mime_data(self, mw, _name, _value):
+ # called by HTMLForm
+ # assert _name == self.name and _value == ''
+ if len(self._upload_data) < 2:
+ if len(self._upload_data) == 0:
+ file_object = BytesIO()
+ content_type = "application/octet-stream"
+ filename = ""
+ else:
+ file_object, content_type, filename = self._upload_data[0]
+ if filename is None:
+ filename = ""
+ mw2 = mw.nextpart()
+ fn_part = '; filename="%s"' % filename
+ disp = 'form-data; name="%s"%s' % (self.name, fn_part)
+ mw2.addheader("Content-Disposition", disp, prefix=1)
+ fh = mw2.startbody(content_type, prefix=0)
+ fh.write(file_object.read())
+ else:
+ # multiple files
+ mw2 = mw.nextpart()
+ disp = 'form-data; name="%s"' % self.name
+ mw2.addheader("Content-Disposition", disp, prefix=1)
+ fh = mw2.startmultipartbody("mixed", prefix=0)
+ for file_object, content_type, filename in self._upload_data:
+ mw3 = mw2.nextpart()
+ if filename is None:
+ filename = ""
+ fn_part = '; filename="%s"' % filename
+ disp = "file%s" % fn_part
+ mw3.addheader("Content-Disposition", disp, prefix=1)
+ fh2 = mw3.startbody(content_type, prefix=0)
+ fh2.write(file_object.read())
+ mw2.lastpart()
+
+ def __str__(self):
+ name = self.name
+ if name is None:
+ name = ""
+
+ if not self._upload_data:
+ value = ""
+ else:
+ value = []
+ for file, ctype, filename in self._upload_data:
+ if filename is None:
+ value.append("")
+ else:
+ value.append(filename)
+ value = ", ".join(value)
+
+ info = []
+ if self.disabled:
+ info.append("disabled")
+ if self.readonly:
+ info.append("readonly")
+ info = ", ".join(info)
+ if info:
+ info = " (%s)" % info
+
+ return "<%s(%s=%s)%s>" % (self.__class__.__name__, name, value, info)
+
+
+# ---------------------------------------------------
+# ---------------------------------------------------
+class IgnoreControl(ScalarControl):
+ """Control that we're not interested in.
+
+ Covers html elements: INPUT/RESET, BUTTON/RESET, INPUT/BUTTON,
+ BUTTON/BUTTON
+
+ These controls are always unsuccessful, in the terminology of HTML 4 (ie.
+ they never require any information to be returned to the server).
+
+ BUTTON/BUTTON is used to generate events for script embedded in HTML.
+
+ The value attribute of IgnoreControl is always None.
+
+ """
+
+ def __init__(self, type, name, attrs, index=None):
+ ScalarControl.__init__(self, type, name, attrs, index)
+ self._value = None
+
+ def is_of_kind(self, kind):
+ return False
+
+ def __setattr__(self, name, value):
+ if name == "value":
+ raise AttributeError("control '%s' is ignored, hence read-only" %
+ self.name)
+ elif name in ("name", "type"):
+ raise AttributeError("%s attribute is readonly" % name)
+ else:
+ self.__dict__[name] = value
+
+
+# ---------------------------------------------------
+# ListControls
+
+# helpers and subsidiary classes
+
+
+class Item:
+ def __init__(self, control, attrs, index=None):
+ label = _get_label(attrs)
+ self.__dict__.update({
+ "name": attrs["value"],
+ "_labels": label and [label] or [],
+ "attrs": attrs,
+ "_control": control,
+ "disabled": 'disabled' in attrs,
+ "_selected": False,
+ "id": attrs.get("id"),
+ "_index": index,
+ })
+ control.items.append(self)
+
+ def get_labels(self):
+ """Return all labels (Label instances) for this item.
+
+ For items that represent radio buttons or checkboxes, if the item was
+ surrounded by a tag, that will be the first label; all other
+ labels, connected by 'for' and 'id', are in the order that appear in
+ the HTML.
+
+ For items that represent select options, if the option had a label
+ attribute, that will be the first label. If the option has contents
+ (text within the option tags) and it is not the same as the label
+ attribute (if any), that will be a label. There is nothing in the
+ spec to my knowledge that makes an option with an id unable to be the
+ target of a label's for attribute, so those are included, if any, for
+ the sake of consistency and completeness.
+
+ """
+ res = []
+ res.extend(self._labels)
+ if self.id:
+ res.extend(self._control._form._id_to_labels.get(self.id, ()))
+ return res
+
+ def __getattr__(self, name):
+ if name == "selected":
+ return self._selected
+ raise AttributeError(name)
+
+ def __setattr__(self, name, value):
+ if name == "selected":
+ self._control._set_selected_state(self, value)
+ elif name == "disabled":
+ self.__dict__["disabled"] = bool(value)
+ else:
+ raise AttributeError(name)
+
+ def __str__(self):
+ res = self.name
+ if self.selected:
+ res = "*" + res
+ if self.disabled:
+ res = "(%s)" % res
+ return res
+
+ def __repr__(self):
+ # XXX appending the attrs without distinguishing them from name and id
+ # is silly
+ attrs = [("name", self.name), ("id", self.id)] + list(
+ iteritems(self.attrs))
+ return "<%s %s>" % (self.__class__.__name__,
+ " ".join(["%s=%r" % (k, v) for k, v in attrs]))
+
+
+def disambiguate(items, nr, **kwds):
+ msgs = []
+ for key, value in iteritems(kwds):
+ msgs.append("%s=%r" % (key, value))
+ msg = " ".join(msgs)
+ if not items:
+ raise ItemNotFoundError(msg)
+ if nr is None:
+ if len(items) > 1:
+ raise AmbiguityError(msg)
+ nr = 0
+ if len(items) <= nr:
+ raise ItemNotFoundError(msg)
+ return items[nr]
+
+
+class ListControl(Control):
+ """Control representing a sequence of items.
+
+ The value attribute of a ListControl represents the successful list items
+ in the control. The successful list items are those that are selected and
+ not disabled.
+
+ ListControl implements both list controls that take a length-1 value
+ (single-selection) and those that take length >1 values
+ (multiple-selection).
+
+ ListControls accept sequence values only. Some controls only accept
+ sequences of length 0 or 1 (RADIO, and single-selection SELECT).
+ In those cases, ItemCountError is raised if len(sequence) > 1. CHECKBOXes
+ and multiple-selection SELECTs (those having the "multiple" HTML attribute)
+ accept sequences of any length.
+
+ Note the following mistake:
+
+ .. code-block:: python
+
+ control.value = some_value
+ assert control.value == some_value # not necessarily true
+
+ The reason for this is that the value attribute always gives the list items
+ in the order they were listed in the HTML.
+
+ ListControl items can also be referred to by their labels instead of names.
+ Use the label argument to .get(), and the .set_value_by_label(),
+ .get_value_by_label() methods.
+
+ Note that, rather confusingly, though SELECT controls are represented in
+ HTML by SELECT elements (which contain OPTION elements, representing
+ individual list items), CHECKBOXes and RADIOs are not represented by *any*
+ element. Instead, those controls are represented by a collection of INPUT
+ elements. For example, this is a SELECT control, named "control1"::
+
+
+ foo
+ bar
+
+
+ and this is a CHECKBOX control, named "control2"::
+
+ ")
+ self._set_selected_state(self._get(name, by_label, nr), selected)
+
+ def _set_selected_state(self, item, action):
+ # action:
+ # bool False: off
+ # bool True: on
+ if self.disabled:
+ raise AttributeError("control '%s' is disabled" % self.name)
+ if self.readonly:
+ raise AttributeError("control '%s' is readonly" % self.name)
+ action == bool(action)
+ if item.disabled:
+ raise AttributeError("item is disabled")
+ if self.multiple:
+ item.__dict__["_selected"] = action
+ else:
+ if not action:
+ item.__dict__["_selected"] = False
+ else:
+ for o in self.items:
+ o.__dict__["_selected"] = False
+ item.__dict__["_selected"] = True
+
+ def toggle_single(self, by_label=None):
+ """Deprecated: toggle the selection of the single item in this control.
+
+ Raises ItemCountError if the control does not contain only one item.
+
+ by_label argument is ignored, and included only for backwards
+ compatibility.
+
+ """
+ deprecation(
+ "control.items[0].selected = not control.items[0].selected")
+ if len(self.items) != 1:
+ raise ItemCountError("'%s' is not a single-item control" %
+ self.name)
+ item = self.items[0]
+ self._set_selected_state(item, not item.selected)
+
+ def set_single(self, selected, by_label=None):
+ """Deprecated: set the selection of the single item in this control.
+
+ Raises ItemCountError if the control does not contain only one item.
+
+ by_label argument is ignored, and included only for backwards
+ compatibility.
+
+ """
+ deprecation("control.items[0].selected = ")
+ if len(self.items) != 1:
+ raise ItemCountError("'%s' is not a single-item control" %
+ self.name)
+ self._set_selected_state(self.items[0], selected)
+
+ def get_item_disabled(self, name, by_label=False, nr=None):
+ """Get disabled state of named list item in a ListControl."""
+ deprecation("control.get(...).disabled")
+ return self._get(name, by_label, nr).disabled
+
+ def set_item_disabled(self, disabled, name, by_label=False, nr=None):
+ """Set disabled state of named list item in a ListControl.
+
+ :arg disabled: boolean disabled state
+
+ """
+ deprecation("control.get(...).disabled = ")
+ self._get(name, by_label, nr).disabled = disabled
+
+ def set_all_items_disabled(self, disabled):
+ """Set disabled state of all list items in a ListControl.
+
+ :arg disabled: boolean disabled state
+
+ """
+ for o in self.items:
+ o.disabled = disabled
+
+ def get_item_attrs(self, name, by_label=False, nr=None):
+ """Return dictionary of HTML attributes for a single ListControl item.
+
+ The HTML element types that describe list items are: OPTION for SELECT
+ controls, INPUT for the rest. These elements have HTML attributes that
+ you may occasionally want to know about -- for example, the "alt" HTML
+ attribute gives a text string describing the item (graphical browsers
+ usually display this as a tooltip).
+
+ The returned dictionary maps HTML attribute names to values. The names
+ and values are taken from the original HTML.
+
+ """
+ deprecation("control.get(...).attrs")
+ return self._get(name, by_label, nr).attrs
+
+ def close_control(self):
+ self._closed = True
+
+ def add_to_form(self, form):
+ assert self._form is None or form == self._form, (
+ "can't add control to more than one form")
+ self._form = form
+ if self.name is None:
+ # always count nameless elements as separate controls
+ Control.add_to_form(self, form)
+ else:
+ for ii in range(len(form.controls) - 1, -1, -1):
+ control = form.controls[ii]
+ if control.name == self.name and control.type == self.type:
+ if control._closed:
+ Control.add_to_form(self, form)
+ else:
+ control.merge_control(self)
+ break
+ else:
+ Control.add_to_form(self, form)
+
+ def merge_control(self, control):
+ assert bool(control.multiple) == bool(self.multiple)
+ # usually, isinstance(control, self.__class__)
+ self.items.extend(control.items)
+
+ def fixup(self):
+ """
+ ListControls are built up from component list items (which are also
+ ListControls) during parsing. This method should be called after all
+ items have been added. See :class:`mechanize.ListControl` for the
+ reason this is required.
+
+ """
+ # Need to set default selection where no item was indicated as being
+ # selected by the HTML:
+
+ # CHECKBOX:
+ # Nothing should be selected.
+ # SELECT/single, SELECT/multiple and RADIO:
+ # RFC 1866 (HTML 2.0): says first item should be selected.
+ # W3C HTML 4.01 Specification: says that client behaviour is
+ # undefined in this case. For RADIO, exactly one must be selected,
+ # though which one is undefined.
+ # Both Netscape and Microsoft Internet Explorer (IE) choose first
+ # item for SELECT/single. However, both IE5 and Mozilla (both 1.0
+ # and Firebird 0.6) leave all items unselected for RADIO and
+ # SELECT/multiple.
+
+ # Since both Netscape and IE all choose the first item for
+ # SELECT/single, we do the same. OTOH, both Netscape and IE
+ # leave SELECT/multiple with nothing selected, in violation of RFC 1866
+ # (but not in violation of the W3C HTML 4 standard); the same is true
+ # of RADIO (which *is* in violation of the HTML 4 standard). We follow
+ # RFC 1866 if the _select_default attribute is set, and Netscape and IE
+ # otherwise. RFC 1866 and HTML 4 are always violated insofar as you
+ # can deselect all items in a RadioControl.
+
+ for o in self.items:
+ # set items' controls to self, now that we've merged
+ o.__dict__["_control"] = self
+
+ def __getattr__(self, name):
+ if name == "value":
+ if self.name is None:
+ return []
+ return [
+ o.name for o in self.items if o.selected and (not o.disabled)
+ ]
+ else:
+ raise AttributeError("%s instance has no attribute '%s'" %
+ (self.__class__.__name__, name))
+
+ def __setattr__(self, name, value):
+ if name == "value":
+ if self.disabled:
+ raise AttributeError("control '%s' is disabled" % self.name)
+ if self.readonly:
+ raise AttributeError("control '%s' is readonly" % self.name)
+ self._set_value(value)
+ elif name in ("name", "type", "multiple"):
+ raise AttributeError("%s attribute is readonly" % name)
+ else:
+ self.__dict__[name] = value
+
+ def _set_value(self, value):
+ if value is None or isstringlike(value):
+ raise TypeError("ListControl, must set a sequence")
+ if not value:
+ for o in self.items:
+ if not o.disabled:
+ o.selected = False
+ elif self.multiple:
+ self._multiple_set_value(value)
+ elif len(value) > 1:
+ raise ItemCountError("single selection list, must set sequence of "
+ "length 0 or 1")
+ else:
+ self._single_set_value(value)
+
+ def _get_items(self, name, target=1):
+ all_items = self.get_items(name)
+ items = [o for o in all_items if not o.disabled]
+ if len(items) < target:
+ if len(all_items) < target:
+ raise ItemNotFoundError("insufficient items with name %r" %
+ name)
+ else:
+ raise AttributeError(
+ "insufficient non-disabled items with name %s" % name)
+ on = []
+ off = []
+ for o in items:
+ if o.selected:
+ on.append(o)
+ else:
+ off.append(o)
+ return on, off
+
+ def _single_set_value(self, value):
+ assert len(value) == 1
+ on, off = self._get_items(value[0])
+ assert len(on) <= 1
+ if not on:
+ off[0].selected = True
+
+ def _multiple_set_value(self, value):
+ turn_on = [] # transactional-ish
+ turn_off = [
+ item for item in self.items
+ if item.selected and (not item.disabled)
+ ]
+ names = {}
+ for nn in value:
+ names[nn] = names.setdefault(nn, 0) + 1
+ for name, count in iteritems(names):
+ on, off = self._get_items(name, count)
+ for i in range(count):
+ if on:
+ item = on[0]
+ del on[0]
+ del turn_off[turn_off.index(item)]
+ else:
+ item = off[0]
+ del off[0]
+ turn_on.append(item)
+ for item in turn_off:
+ item.selected = False
+ for item in turn_on:
+ item.selected = True
+
+ def set_value_by_label(self, value):
+ """Set the value of control by item labels.
+
+ value is expected to be an iterable of strings that are substrings of
+ the item labels that should be selected. Before substring matching is
+ performed, the original label text is whitespace-compressed
+ (consecutive whitespace characters are converted to a single space
+ character) and leading and trailing whitespace is stripped. Ambiguous
+ labels: it will not complain as long as all ambiguous labels share the
+ same item name (e.g. OPTION value).
+
+ """
+ if isstringlike(value):
+ raise TypeError(value)
+ if not self.multiple and len(value) > 1:
+ raise ItemCountError("single selection list, must set sequence of "
+ "length 0 or 1")
+ items = []
+ for nn in value:
+ found = self.get_items(label=nn)
+ if len(found) > 1:
+ # ambiguous labels are fine as long as item names (e.g.
+ # OPTION values) are same
+ opt_name = found[0].name
+ if [o for o in found[1:] if o.name != opt_name]:
+ raise AmbiguityError(nn)
+ for o in found:
+ # For the multiple-item case, we could try to be smarter,
+ # saving them up and trying to resolve, but that's too much.
+ if o not in items:
+ items.append(o)
+ break
+ else: # all of them are used
+ raise ItemNotFoundError(nn)
+ # now we have all the items that should be on
+ # let's just turn everything off and then back on.
+ self.value = []
+ for o in items:
+ o.selected = True
+
+ def get_value_by_label(self):
+ """Return the value of the control as given by normalized labels."""
+ res = []
+ for o in self.items:
+ if (not o.disabled) and o.selected:
+ for l in o.get_labels():
+ if l.text:
+ res.append(l.text)
+ break
+ else:
+ res.append(None)
+ return res
+
+ def possible_items(self, by_label=False):
+ """Deprecated: return the names or labels of all possible items.
+
+ Includes disabled items, which may be misleading for some use cases.
+
+ """
+ deprecation("[item.name for item in self.items]")
+ if by_label:
+ res = []
+ for o in self.items:
+ for l in o.get_labels():
+ if l.text:
+ res.append(l.text)
+ break
+ else:
+ res.append(None)
+ return res
+ return [o.name for o in self.items]
+
+ def _totally_ordered_pairs(self):
+ if self.disabled or self.name is None:
+ return []
+ else:
+ return [(o._index, self.name, o.name) for o in self.items
+ if o.selected and not o.disabled]
+
+ def __str__(self):
+ name = self.name
+ if name is None:
+ name = ""
+
+ display = [str(o) for o in self.items]
+
+ infos = []
+ if self.disabled:
+ infos.append("disabled")
+ if self.readonly:
+ infos.append("readonly")
+ info = ", ".join(infos)
+ if info:
+ info = " (%s)" % info
+
+ return "<%s(%s=[%s])%s>" % (self.__class__.__name__, name,
+ ", ".join(display), info)
+
+
+class RadioControl(ListControl):
+ """
+ Covers:
+
+ INPUT/RADIO
+
+ """
+
+ def __init__(self, type, name, attrs, select_default=False, index=None):
+ attrs.setdefault("value", "on")
+ ListControl.__init__(
+ self,
+ type,
+ name,
+ attrs,
+ select_default,
+ called_as_base_class=True,
+ index=index)
+ self.__dict__["multiple"] = False
+ o = Item(self, attrs, index)
+ o.__dict__["_selected"] = 'checked' in attrs
+
+ def fixup(self):
+ ListControl.fixup(self)
+ found = [o for o in self.items if o.selected and not o.disabled]
+ if not found:
+ if self._select_default:
+ for o in self.items:
+ if not o.disabled:
+ o.selected = True
+ break
+ else:
+ # Ensure only one item selected. Choose the last one,
+ # following IE and Firefox.
+ for o in found[:-1]:
+ o.selected = False
+
+ def get_labels(self):
+ return []
+
+
+class CheckboxControl(ListControl):
+ """
+ Covers:
+
+ INPUT/CHECKBOX
+
+ """
+
+ def __init__(self, type, name, attrs, select_default=False, index=None):
+ attrs.setdefault("value", "on")
+ ListControl.__init__(
+ self,
+ type,
+ name,
+ attrs,
+ select_default,
+ called_as_base_class=True,
+ index=index)
+ self.__dict__["multiple"] = True
+ o = Item(self, attrs, index)
+ o.__dict__["_selected"] = 'checked' in attrs
+
+ def get_labels(self):
+ return []
+
+
+class SelectControl(ListControl):
+ """
+ Covers:
+
+ SELECT (and OPTION)
+
+
+ OPTION 'values', in HTML parlance, are Item 'names' in mechanize parlance.
+
+ SELECT control values and labels are subject to some messy defaulting
+ rules. For example, if the HTML representation of the control is::
+
+
+ current year
+ 2001
+ 2000
+
+
+ The items, in order, have labels "2002", "2001" and "2000", whereas their
+ names (the OPTION values) are "0", "1" and "2000" respectively. Note that
+ the value of the last OPTION in this example defaults to its contents, as
+ specified by RFC 1866, as do the labels of the second and third OPTIONs.
+
+ The OPTION labels are sometimes more meaningful than the OPTION values,
+ which can make for more maintainable code.
+
+ Additional read-only public attribute: attrs
+
+ The attrs attribute is a dictionary of the original HTML attributes of the
+ SELECT element. Other ListControls do not have this attribute, because in
+ other cases the control as a whole does not correspond to any single HTML
+ element. control.get(...).attrs may be used as usual to get at the HTML
+ attributes of the HTML elements corresponding to individual list items (for
+ SELECT controls, these are OPTION elements).
+
+ Another special case is that the Item.attrs dictionaries have a special key
+ "contents" which does not correspond to any real HTML attribute, but rather
+ contains the contents of the OPTION element::
+
+ this bit
+
+ """
+
+ # HTML attributes here are treated slightly differently from other list
+ # controls:
+ # -The SELECT HTML attributes dictionary is stuffed into the OPTION
+ # HTML attributes dictionary under the "__select" key.
+ # -The content of each OPTION element is stored under the special
+ # "contents" key of the dictionary.
+ # After all this, the dictionary is passed to the SelectControl constructor
+ # as the attrs argument, as usual. However:
+ # -The first SelectControl constructed when building up a SELECT control
+ # has a constructor attrs argument containing only the __select key -- so
+ # this SelectControl represents an empty SELECT control.
+ # -Subsequent SelectControls have both OPTION HTML-attribute in attrs and
+ # the __select dictionary containing the SELECT HTML-attributes.
+
+ def __init__(self, type, name, attrs, select_default=False, index=None):
+ # fish out the SELECT HTML attributes from the OPTION HTML attributes
+ # dictionary
+ self.attrs = dict(attrs["__select"])
+ self.__dict__["_label"] = _get_label(self.attrs)
+ self.__dict__["id"] = self.attrs.get("id")
+ self.__dict__["multiple"] = 'multiple' in self.attrs
+ # the majority of the contents, label, and value dance already happened
+ contents = attrs.get("contents")
+ attrs = dict(attrs)
+ del attrs["__select"]
+
+ ListControl.__init__(
+ self,
+ type,
+ name,
+ self.attrs,
+ select_default,
+ called_as_base_class=True,
+ index=index)
+ self.disabled = 'disabled' in self.attrs
+ self.readonly = 'readonly' in self.attrs
+ if 'value' in attrs:
+ # otherwise it is a marker 'select started' token
+ o = Item(self, attrs, index)
+ o.__dict__["_selected"] = 'selected' in attrs
+ # add 'label' label and contents label, if different. If both are
+ # provided, the 'label' label is used for display in HTML
+ # 4.0-compliant browsers (and any lower spec? not sure) while the
+ # contents are used for display in older or less-compliant
+ # browsers. We make label objects for both, if the values are
+ # different.
+ label = attrs.get("label")
+ if label:
+ o._labels.append(Label(label))
+ if contents and contents != label:
+ o._labels.append(Label(contents))
+ elif contents:
+ o._labels.append(Label(contents))
+
+ def fixup(self):
+ ListControl.fixup(self)
+ # Firefox doesn't exclude disabled items from those considered here
+ # (i.e. from 'found', for both branches of the if below). Note that
+ # IE6 doesn't support the disabled attribute on OPTIONs at all.
+ found = [o for o in self.items if o.selected]
+ if not found:
+ if not self.multiple or self._select_default:
+ for o in self.items:
+ if not o.disabled:
+ was_disabled = self.disabled
+ self.disabled = False
+ try:
+ o.selected = True
+ finally:
+ o.disabled = was_disabled
+ break
+ elif not self.multiple:
+ # Ensure only one item selected. Choose the last one,
+ # following IE and Firefox.
+ for o in found[:-1]:
+ o.selected = False
+
+
+# ---------------------------------------------------
+class SubmitControl(ScalarControl):
+ """
+ Covers:
+
+ INPUT/SUBMIT
+ BUTTON/SUBMIT
+
+ """
+
+ def __init__(self, type, name, attrs, index=None):
+ ScalarControl.__init__(self, type, name, attrs, index)
+ # IE5 defaults SUBMIT value to "Submit Query"; Firebird 0.6 leaves it
+ # blank, Konqueror 3.1 defaults to "Submit". HTML spec. doesn't seem
+ # to define this.
+ if self.value is None:
+ self.__dict__['_value'] = ""
+ self.readonly = True
+
+ def get_labels(self):
+ res = []
+ if self.value:
+ res.append(Label(self.value))
+ res.extend(ScalarControl.get_labels(self))
+ return res
+
+ def is_of_kind(self, kind):
+ return kind == "clickable"
+
+ def _click(self, form, coord, return_type, request_class=_request.Request):
+ self._clicked = coord
+ r = form._switch_click(return_type, request_class)
+ self._clicked = False
+ return r
+
+ def _totally_ordered_pairs(self):
+ if not self._clicked:
+ return []
+ return ScalarControl._totally_ordered_pairs(self)
+
+
+# ---------------------------------------------------
+class ImageControl(SubmitControl):
+ """
+ Covers:
+
+ INPUT/IMAGE
+
+ Coordinates are specified using one of the HTMLForm.click* methods.
+
+ """
+
+ def __init__(self, type, name, attrs, index=None):
+ SubmitControl.__init__(self, type, name, attrs, index)
+ self.readonly = False
+
+ def _totally_ordered_pairs(self):
+ clicked = self._clicked
+ if self.disabled or not clicked:
+ return []
+ name = self.name
+ if name is None:
+ return []
+ pairs = [
+ (self._index, "%s.x" % name, str(clicked[0])),
+ (self._index + 1, "%s.y" % name, str(clicked[1])),
+ ]
+ value = self._value
+ if value:
+ pairs.append((self._index + 2, name, value))
+ return pairs
+
+ get_labels = ScalarControl.get_labels
+
+
+# aliases, just to make str(control) and str(form) clearer
+class PasswordControl(TextControl):
+ pass
+
+
+class HiddenControl(TextControl):
+ pass
+
+
+class TextareaControl(TextControl):
+ pass
+
+
+class SubmitButtonControl(SubmitControl):
+ pass
+
+
+def is_listcontrol(control):
+ return control.is_of_kind("list")
+
+
+class HTMLForm:
+ """
+ Represents a single HTML element.
+
+ A form consists of a sequence of controls that usually have names, and
+ which can take on various values. The values of the various types of
+ controls represent variously: text, zero-or-one-of-many or many-of-many
+ choices, and files to be uploaded. Some controls can be clicked on to
+ submit the form, and clickable controls' values sometimes include the
+ coordinates of the click.
+
+ Forms can be filled in with data to be returned to the server, and then
+ submitted, using the click method to generate a request object suitable for
+ passing to :func:`mechanize.urlopen` (or the click_request_data or
+ click_pairs methods for integration with third-party code).
+
+ Usually, HTMLForm instances are not created directly. Instead, they are
+ automatically created when visting a page with a mechanize Browser. If you
+ do construct HTMLForm objects yourself, however, note that an HTMLForm
+ instance is only properly initialised after the fixup method has been
+ called. See :class:`mechanize.ListControl` for the reason this is
+ required.
+
+ Indexing a form (form["control_name"]) returns the named Control's value
+ attribute. Assignment to a form index (form["control_name"] = something)
+ is equivalent to assignment to the named Control's value attribute. If you
+ need to be more specific than just supplying the control's name, use the
+ set_value and get_value methods.
+
+ ListControl values are lists of item names (specifically, the names of the
+ items that are selected and not disabled, and hence are "successful" -- ie.
+ cause data to be returned to the server). The list item's name is the
+ value of the corresponding HTML element's"value" attribute.
+
+ Example::
+
+
+ 1
+ cheddar
+
+
+ defines a SELECT control with name "more_cheeses" which has two items,
+ named "1" and "2" (because the OPTION element's value HTML attribute
+ defaults to the element contents -- see :class:`mechanize.SelectControl`
+ for more on these defaulting rules).
+
+ To select, deselect or otherwise manipulate individual list items, use the
+ :meth:`mechanize.HTMLForm.find_control()` and
+ :meth:`mechanize.ListControl.get()` methods. To set the whole value, do as
+ for any other control: use indexing or the `set_value/get_value` methods.
+
+ Example:
+
+ .. code-block:: python
+
+ # select *only* the item named "cheddar"
+ form["cheeses"] = ["cheddar"]
+ # select "cheddar", leave other items unaffected
+ form.find_control("cheeses").get("cheddar").selected = True
+
+ Some controls (RADIO and SELECT without the multiple attribute) can only
+ have zero or one items selected at a time. Some controls (CHECKBOX and
+ SELECT with the multiple attribute) can have multiple items selected at a
+ time. To set the whole value of a ListControl, assign a sequence to a form
+ index:
+
+ .. code-block:: python
+
+ form["cheeses"] = ["cheddar", "leicester"]
+
+ If the ListControl is not multiple-selection, the assigned list must be of
+ length one.
+
+ To check if a control has an item, if an item is selected, or if an item is
+ successful (selected and not disabled), respectively:
+
+ .. code-block:: python
+
+ "cheddar" in [item.name for item in form.find_control("cheeses").items]
+ "cheddar" in [item.name for item in form.find_control("cheeses").items
+ and item.selected]
+ "cheddar" in form["cheeses"]
+ # or
+ "cheddar" in form.get_value("cheeses")
+
+ Note that some list items may be disabled (see below).
+
+ Note the following mistake:
+
+ .. code-block:: python
+
+ form[control_name] = control_value
+ assert form[control_name] == control_value # not necessarily true
+
+ The reason for this is that form[control_name] always gives the list items
+ in the order they were listed in the HTML.
+
+ List items (hence list values, too) can be referred to in terms of list
+ item labels rather than list item names using the appropriate label
+ arguments. Note that each item may have several labels.
+
+ The question of default values of OPTION contents, labels and values is
+ somewhat complicated: see :class:`mechanize.SelectControl` and
+ :meth:`mechanize.ListControl.get_item_attrs()` if you think you need to
+ know.
+
+ Controls can be disabled or readonly. In either case, the control's value
+ cannot be changed until you clear those flags (see example below).
+ Disabled is the state typically represented by browsers by 'greying out' a
+ control. Disabled controls are not 'successful' -- they don't cause data
+ to get returned to the server. Readonly controls usually appear in
+ browsers as read-only text boxes. Readonly controls are successful. List
+ items can also be disabled. Attempts to select or deselect disabled items
+ fail with AttributeError.
+
+ If a lot of controls are readonly, it can be useful to do this:
+
+ .. code-block:: python
+
+ form.set_all_readonly(False)
+
+ To clear a control's value attribute, so that it is not successful (until a
+ value is subsequently set):
+
+ .. code-block:: python
+
+ form.clear("cheeses")
+
+ More examples:
+
+ .. code-block:: python
+
+ control = form.find_control("cheeses")
+ control.disabled = False
+ control.readonly = False
+ control.get("gruyere").disabled = True
+ control.items[0].selected = True
+
+ See the various Control classes for further documentation. Many methods
+ take name, type, kind, id, label and nr arguments to specify the control to
+ be operated on: see :meth:`mechanize.HTMLForm.find_control()`.
+
+ ControlNotFoundError (subclass of ValueError) is raised if the specified
+ control can't be found. This includes occasions where a non-ListControl
+ is found, but the method (set, for example) requires a ListControl.
+ ItemNotFoundError (subclass of ValueError) is raised if a list item can't
+ be found. ItemCountError (subclass of ValueError) is raised if an attempt
+ is made to select more than one item and the control doesn't allow that, or
+ set/get_single are called and the control contains more than one item.
+ AttributeError is raised if a control or item is readonly or disabled and
+ an attempt is made to alter its value.
+
+ Security note: Remember that any passwords you store in HTMLForm instances
+ will be saved to disk in the clear if you pickle them (directly or
+ indirectly). The simplest solution to this is to avoid pickling HTMLForm
+ objects. You could also pickle before filling in any password, or just set
+ the password to "" before pickling.
+
+
+ Public attributes:
+
+ :ivar action: full (absolute URI) form action
+ :ivar method: "GET" or "POST"
+ :ivar enctype: form transfer encoding MIME type
+ :ivar name: name of form (None if no name was specified)
+ :ivar attrs: dictionary mapping original HTML form attributes to their
+ values
+ :ivar controls: list of Control instances; do not alter this list
+ (instead, call form.new_control to make a Control and add it to the
+ form, or control.add_to_form if you already have a Control instance)
+
+
+
+ Methods for form filling:
+
+ Most of the these methods have very similar arguments. See
+ :meth:`mechanize.HTMLForm.find_control()` for details of the name, type,
+ kind, label and nr arguments.
+
+ .. code-block:: python
+
+ def find_control(self,
+ name=None, type=None, kind=None, id=None,
+ predicate=None, nr=None, label=None)
+
+ get_value(name=None, type=None, kind=None, id=None, nr=None,
+ by_label=False, # by_label is deprecated
+ label=None)
+ set_value(value,
+ name=None, type=None, kind=None, id=None, nr=None,
+ by_label=False, # by_label is deprecated
+ label=None)
+
+ clear_all()
+ clear(name=None, type=None, kind=None, id=None, nr=None, label=None)
+
+ set_all_readonly(readonly)
+
+
+ Method applying only to FileControls:
+
+ .. code-block:: python
+
+ add_file(file_object,
+ content_type="application/octet-stream", filename=None,
+ name=None, id=None, nr=None, label=None)
+
+
+ Methods applying only to clickable controls:
+
+ .. code-block:: python
+
+ click(name=None, type=None, id=None, nr=0, coord=(1,1), label=None)
+ click_request_data(name=None, type=None, id=None, nr=0, coord=(1,1),
+ label=None)
+ click_pairs(name=None, type=None, id=None, nr=0, coord=(1,1),
+ label=None)
+
+ """
+
+ type2class = {
+ "text": TextControl,
+ "password": PasswordControl,
+ "hidden": HiddenControl,
+ "textarea": TextareaControl,
+ "file": FileControl,
+ "button": IgnoreControl,
+ "buttonbutton": IgnoreControl,
+ "reset": IgnoreControl,
+ "resetbutton": IgnoreControl,
+ "submit": SubmitControl,
+ "submitbutton": SubmitButtonControl,
+ "image": ImageControl,
+ "radio": RadioControl,
+ "checkbox": CheckboxControl,
+ "select": SelectControl,
+ }
+
+ # ---------------------------------------------------
+ # Initialisation. Use ParseResponse / ParseFile instead.
+
+ def __init__(self,
+ action,
+ method="GET",
+ enctype="application/x-www-form-urlencoded",
+ name=None,
+ attrs=None,
+ request_class=_request.Request,
+ forms=None,
+ labels=None,
+ id_to_labels=None,
+ encoding=None):
+ """
+ In the usual case, use ParseResponse (or ParseFile) to create new
+ HTMLForm objects.
+
+ action: full (absolute URI) form action
+ method: "GET" or "POST"
+ enctype: form transfer encoding MIME type
+ name: name of form
+ attrs: dictionary mapping original HTML form attributes to their values
+
+ """
+ self.action = action
+ self.method = method
+ self.enctype = enctype
+ self.form_encoding = encoding or 'utf-8'
+ self.name = name
+ if attrs is not None:
+ self.attrs = dict(attrs)
+ else:
+ self.attrs = {}
+ self.controls = []
+ self._request_class = request_class
+
+ # these attributes are used by zope.testbrowser
+ self._forms = forms # this is a semi-public API!
+ self._labels = labels # this is a semi-public API!
+ self._id_to_labels = id_to_labels # this is a semi-public API!
+
+ self._urlunparse = urlunparse
+ self._urlparse = urlparse
+
+ def new_control(self,
+ type,
+ name,
+ attrs,
+ ignore_unknown=False,
+ select_default=False,
+ index=None):
+ """Adds a new control to the form.
+
+ This is usually called by mechanize. Don't call it
+ yourself unless you're building your own Control instances.
+
+ Note that controls representing lists of items are built up from
+ controls holding only a single list item. See
+ :class:`mechanize.ListControl` for further information.
+
+ :arg type: type of control (see :class:`mechanize.Control` for a list)
+ :arg attrs: HTML attributes of control
+ :arg ignore_unknown: if true, use a dummy Control instance for controls
+ of unknown type; otherwise, use a TextControl
+ :arg select_default: for RADIO and multiple-selection SELECT controls,
+ pick the first item as the default if no 'selected' HTML attribute
+ is present (this defaulting happens when the HTMLForm.fixup method
+ is called)
+ :arg index: index of corresponding element in HTML (see
+ MoreFormTests.test_interspersed_controls for motivation)
+
+ """
+ type = type.lower()
+ klass = self.type2class.get(type)
+ if klass is None:
+ if ignore_unknown:
+ klass = IgnoreControl
+ else:
+ klass = TextControl
+
+ a = dict(attrs)
+ if issubclass(klass, ListControl):
+ control = klass(type, name, a, select_default, index)
+ else:
+ control = klass(type, name, a, index)
+
+ if type == "select" and len(attrs) == 1:
+ for ii in range(len(self.controls) - 1, -1, -1):
+ ctl = self.controls[ii]
+ if ctl.type == "select":
+ ctl.close_control()
+ break
+
+ control.add_to_form(self)
+ control._urlparse = self._urlparse
+ control._urlunparse = self._urlunparse
+
+ def fixup(self):
+ """Normalise form after all controls have been added.
+
+ This is usually called by ParseFile and ParseResponse. Don't call it
+ youself unless you're building your own Control instances.
+
+ This method should only be called once, after all controls have been
+ added to the form.
+
+ """
+ for control in self.controls:
+ control.fixup()
+ control.form_encoding = self.form_encoding
+
+# ---------------------------------------------------
+
+ def __str__(self):
+ header = "%s%s %s %s" % ((self.name and self.name + " " or ""),
+ self.method, self.action, self.enctype)
+ rep = [header]
+ for control in self.controls:
+ rep.append(" %s" % str(control))
+ return "<%s>" % "\n".join(rep)
+
+# ---------------------------------------------------
+# Form-filling methods.
+
+ def __getitem__(self, name):
+ return self.find_control(name).value
+
+ def __contains__(self, name):
+ return bool(self.find_control(name))
+
+ def __setitem__(self, name, value):
+ control = self.find_control(name)
+ try:
+ control.value = value
+ except AttributeError as e:
+ raise ValueError(str(e))
+
+ def get_value(
+ self,
+ name=None,
+ type=None,
+ kind=None,
+ id=None,
+ nr=None,
+ by_label=False, # by_label is deprecated
+ label=None):
+ """Return value of control.
+
+ If only name and value arguments are supplied, equivalent to
+
+ .. code-block:: python
+
+ form[name]
+
+ """
+ if by_label:
+ deprecation("form.get_value_by_label(...)")
+ c = self.find_control(name, type, kind, id, label=label, nr=nr)
+ if by_label:
+ try:
+ meth = c.get_value_by_label
+ except AttributeError:
+ raise NotImplementedError(
+ "control '%s' does not yet support by_label" % c.name)
+ else:
+ return meth()
+ else:
+ return c.value
+
+ def set_value(
+ self,
+ value,
+ name=None,
+ type=None,
+ kind=None,
+ id=None,
+ nr=None,
+ by_label=False, # by_label is deprecated
+ label=None):
+ """Set value of control.
+
+ If only name and value arguments are supplied, equivalent to
+
+ .. code-block:: python
+
+ form[name] = value
+
+ """
+ if by_label:
+ deprecation("form.get_value_by_label(...)")
+ c = self.find_control(name, type, kind, id, label=label, nr=nr)
+ if by_label:
+ try:
+ meth = c.set_value_by_label
+ except AttributeError:
+ raise NotImplementedError(
+ "control '%s' does not yet support by_label" % c.name)
+ else:
+ meth(value)
+ else:
+ c.value = value
+
+ def get_value_by_label(self,
+ name=None,
+ type=None,
+ kind=None,
+ id=None,
+ label=None,
+ nr=None):
+ """
+
+ All arguments should be passed by name.
+
+ """
+ c = self.find_control(name, type, kind, id, label=label, nr=nr)
+ return c.get_value_by_label()
+
+ def set_value_by_label(self,
+ value,
+ name=None,
+ type=None,
+ kind=None,
+ id=None,
+ label=None,
+ nr=None):
+ """
+
+ All arguments should be passed by name.
+
+ """
+ c = self.find_control(name, type, kind, id, label=label, nr=nr)
+ c.set_value_by_label(value)
+
+ def set_all_readonly(self, readonly):
+ for control in self.controls:
+ control.readonly = bool(readonly)
+
+ def clear_all(self):
+ """Clear the value attributes of all controls in the form.
+
+ See :meth:`mechanize.HTMLForm.clear()`
+
+ """
+ for control in self.controls:
+ control.clear()
+
+ def clear(self,
+ name=None,
+ type=None,
+ kind=None,
+ id=None,
+ nr=None,
+ label=None):
+ """Clear the value attribute of a control.
+
+ As a result, the affected control will not be successful until a value
+ is subsequently set. AttributeError is raised on readonly controls.
+
+ """
+ c = self.find_control(name, type, kind, id, label=label, nr=nr)
+ c.clear()
+
+# ---------------------------------------------------
+# Form-filling methods applying only to ListControls.
+
+ def possible_items(
+ self, # deprecated
+ name=None,
+ type=None,
+ kind=None,
+ id=None,
+ nr=None,
+ by_label=False,
+ label=None):
+ """Return a list of all values that the specified control can take."""
+ c = self._find_list_control(name, type, kind, id, label, nr)
+ return c.possible_items(by_label)
+
+ def set(
+ self,
+ selected,
+ item_name, # deprecated
+ name=None,
+ type=None,
+ kind=None,
+ id=None,
+ nr=None,
+ by_label=False,
+ label=None):
+ """Select / deselect named list item.
+
+ :arg selected: boolean selected state
+
+ """
+ self._find_list_control(name, type, kind, id, label, nr).set(
+ selected, item_name, by_label)
+
+ def toggle(
+ self,
+ item_name, # deprecated
+ name=None,
+ type=None,
+ kind=None,
+ id=None,
+ nr=None,
+ by_label=False,
+ label=None):
+ """Toggle selected state of named list item."""
+ self._find_list_control(name, type, kind, id, label, nr).toggle(
+ item_name, by_label)
+
+ def set_single(
+ self,
+ selected, # deprecated
+ name=None,
+ type=None,
+ kind=None,
+ id=None,
+ nr=None,
+ by_label=None,
+ label=None):
+ """Select / deselect list item in a control having only one item.
+
+ If the control has multiple list items, ItemCountError is raised.
+
+ This is just a convenience method, so you don't need to know the item's
+ name -- the item name in these single-item controls is usually
+ something meaningless like "1" or "on".
+
+ For example, if a checkbox has a single item named "on", the following
+ two calls are equivalent:
+
+ .. code-block:: python
+
+ control.toggle("on")
+ control.toggle_single()
+
+ """ # by_label ignored and deprecated
+ self._find_list_control(name, type, kind, id, label,
+ nr).set_single(selected)
+
+ def toggle_single(self,
+ name=None,
+ type=None,
+ kind=None,
+ id=None,
+ nr=None,
+ by_label=None,
+ label=None): # deprecated
+ """Toggle selected state of list item in control having only one item.
+
+ The rest is as for :meth:`mechanize.HTMLForm.set_single()`
+
+ """ # by_label ignored and deprecated
+ self._find_list_control(name, type, kind, id, label,
+ nr).toggle_single()
+
+# ---------------------------------------------------
+# Form-filling method applying only to FileControls.
+
+ def add_file(self,
+ file_object,
+ content_type=None,
+ filename=None,
+ name=None,
+ id=None,
+ nr=None,
+ label=None):
+ """Add a file to be uploaded.
+
+ :arg file_object: file-like object (with read method) from which to
+ read data to upload
+ :arg content_type: MIME content type of data to upload
+ :arg filename: filename to pass to server
+
+ If filename is None, no filename is sent to the server.
+
+ If content_type is None, the content type is guessed based on the
+ filename and the data from read from the file object.
+
+ At the moment, guessed content type is always application/octet-stream.
+
+ Note the following useful HTML attributes of file upload controls (see
+ HTML 4.01 spec, section 17):
+
+ * `accept`: comma-separated list of content types
+ that the server will handle correctly;
+ you can use this to filter out non-conforming files
+ * `size`: XXX IIRC, this is indicative of whether form
+ wants multiple or single files
+ * `maxlength`: XXX hint of max content length in bytes?
+
+ """
+ self.find_control(
+ name, "file", id=id, label=label,
+ nr=nr).add_file(file_object, content_type, filename)
+
+# ---------------------------------------------------
+# Form submission methods, applying only to clickable controls.
+
+ def click(self,
+ name=None,
+ type=None,
+ id=None,
+ nr=0,
+ coord=(1, 1),
+ request_class=_request.Request,
+ label=None):
+ """Return request that would result from clicking on a control.
+
+ The request object is a mechanize.Request instance, which you can pass
+ to mechanize.urlopen.
+
+ Only some control types (INPUT/SUBMIT & BUTTON/SUBMIT buttons and
+ IMAGEs) can be clicked.
+
+ Will click on the first clickable control, subject to the name, type
+ and nr arguments (as for find_control). If no name, type, id or number
+ is specified and there are no clickable controls, a request will be
+ returned for the form in its current, un-clicked, state.
+
+ IndexError is raised if any of name, type, id or nr is specified but no
+ matching control is found. ValueError is raised if the HTMLForm has an
+ enctype attribute that is not recognised.
+
+ You can optionally specify a coordinate to click at, which only makes a
+ difference if you clicked on an image.
+
+ """
+ return self._click(name, type, id, label, nr, coord, "request",
+ self._request_class)
+
+ def click_request_data(self,
+ name=None,
+ type=None,
+ id=None,
+ nr=0,
+ coord=(1, 1),
+ request_class=_request.Request,
+ label=None):
+ """As for click method, but return a tuple (url, data, headers).
+
+ You can use this data to send a request to the server. This is useful
+ if you're using httplib or urllib rather than mechanize. Otherwise,
+ use the click method.
+
+ """
+ return self._click(name, type, id, label, nr, coord, "request_data",
+ self._request_class)
+
+ def click_pairs(self,
+ name=None,
+ type=None,
+ id=None,
+ nr=0,
+ coord=(1, 1),
+ label=None):
+ """As for click_request_data, but returns a list of (key, value) pairs.
+
+ You can use this list as an argument to urllib.urlencode. This is
+ usually only useful if you're using httplib or urllib rather than
+ mechanize. It may also be useful if you want to manually tweak the
+ keys and/or values, but this should not be necessary. Otherwise, use
+ the click method.
+
+ Note that this method is only useful for forms of MIME type
+ x-www-form-urlencoded. In particular, it does not return the
+ information required for file upload. If you need file upload and are
+ not using mechanize, use click_request_data.
+ """
+ return self._click(name, type, id, label, nr, coord, "pairs",
+ self._request_class)
+
+# ---------------------------------------------------
+
+ def find_control(self,
+ name=None,
+ type=None,
+ kind=None,
+ id=None,
+ predicate=None,
+ nr=None,
+ label=None):
+ """Locate and return some specific control within the form.
+
+ At least one of the name, type, kind, predicate and nr arguments must
+ be supplied. If no matching control is found, ControlNotFoundError is
+ raised.
+
+ If name is specified, then the control must have the indicated name.
+
+ If type is specified then the control must have the specified type (in
+ addition to the types possible for sz and sz > -1:
+ ans, self.__data = ans[:sz], ans[sz:]
+ return ans
+
+ def readline(self, sz=-1):
+ # Dont care about making this efficient
+ data = self.read()
+ idx = data.find(b'\n')
+ if idx > 0:
+ if sz < 0 or idx < sz:
+ line, self.__data = data[:idx + 1], data[idx + 1:]
+ else:
+ line, self.__data = data[:sz], data[sz:]
+ else:
+ if sz > -1:
+ line, self.__data = data[:sz], data[sz:]
+ else:
+ line = data
+ return line
+
+ def close(self):
+ self.__fp.close()
+
+ def fileno(self):
+ return self.__fp.fileno()
+
+ def __iter__(self):
+ ans = self.readline()
+ if ans:
+ yield ans
+
+ def next(self):
+ ans = self.readline()
+ if not ans:
+ raise StopIteration()
+ return ans
+
+
+def create_gzip_decompressor(zipped_file):
+ prefix = read_amt(zipped_file, 10)
+ if prefix[:2] != b'\x1f\x8b':
+ raise ValueError('gzip stream has incorrect magic bytes: %r' %
+ prefix[:2])
+ if prefix[2:3] != b'\x08':
+ raise ValueError('gzip stream has unknown compression method: %r' %
+ prefix[2])
+ flag = ord(prefix[3:4])
+ if flag & 4: # extra
+ extra_amt = read_amt(zipped_file, 2)
+ extra_amt = ord(extra_amt[0]) + 256 * ord(extra_amt[1])
+ if extra_amt:
+ read_amt(zipped_file, extra_amt)
+ if flag & 8: # filename
+ while read_amt(zipped_file, 1) != b'\0':
+ continue
+ if flag & 16: # comment
+ while read_amt(zipped_file, 1) != b'\0':
+ continue
+ if flag & 2: # crc
+ read_amt(zipped_file, 2)
+ return UnzipWrapper(zipped_file)
+
+
+class HTTPGzipProcessor(BaseHandler):
+ handler_order = 200 # response processing before HTTPEquivProcessor
+
+ def __init__(self, request_gzip=False):
+ self.request_gzip = request_gzip
+
+ def __copy__(self):
+ return self.__class__(self.request_gzip)
+
+ def http_request(self, request):
+ if self.request_gzip:
+ existing = [
+ x.strip()
+ for x in request.get_header('Accept-Encoding', '').split(',')
+ ]
+ if sum('gzip' in x for x in existing) < 1:
+ existing.append('gzip')
+ request.add_header("Accept-Encoding",
+ ', '.join(filter(None, existing)))
+ return request
+
+ def http_response(self, request, response):
+ # post-process response
+ h = response.info()
+ enc_hdrs = h.getheaders("Content-encoding")
+ for enc_hdr in enc_hdrs:
+ if "gzip" in enc_hdr:
+ response._set_fp(create_gzip_decompressor(response.fp))
+ del h['Content-encoding']
+ del h['Content-length']
+ return response
+
+ https_response = http_response
+ https_request = http_request
diff --git a/core/lib/mechanize/_headersutil.py b/core/lib/mechanize/_headersutil.py
new file mode 100644
index 00000000..9a00fb62
--- /dev/null
+++ b/core/lib/mechanize/_headersutil.py
@@ -0,0 +1,278 @@
+"""Utility functions for HTTP header value parsing and construction.
+
+Copyright 1997-1998, Gisle Aas
+Copyright 2002-2006, John J. Lee
+
+This code is free software; you can redistribute it and/or modify it
+under the terms of the BSD or ZPL 2.1 licenses (see the file
+LICENSE included with the distribution).
+
+"""
+
+from __future__ import absolute_import
+
+import os
+import re
+
+from . import _rfc3986
+from ._util import http2time
+from .polyglot import is_string
+
+
+def is_html_file_extension(url, allow_xhtml):
+ ext = os.path.splitext(_rfc3986.urlsplit(url)[2])[1]
+ html_exts = [".htm", ".html"]
+ if allow_xhtml:
+ html_exts += [".xhtml"]
+ return ext in html_exts
+
+
+def is_html(ct_headers, url, allow_xhtml=False):
+ """
+ ct_headers: Sequence of Content-Type headers
+ url: Response URL
+
+ """
+ if not ct_headers:
+ return is_html_file_extension(url, allow_xhtml)
+ headers = split_header_words(ct_headers)
+ if len(headers) < 1:
+ return is_html_file_extension(url, allow_xhtml)
+ first_header = headers[0]
+ first_parameter = first_header[0]
+ ct = first_parameter[0]
+ html_types = ["text/html"]
+ if allow_xhtml:
+ html_types += [
+ "text/xhtml",
+ "text/xml",
+ "application/xml",
+ "application/xhtml+xml",
+ ]
+ return ct in html_types
+
+
+def unmatched(match):
+ """Return unmatched part of re.Match object."""
+ start, end = match.span(0)
+ return match.string[:start] + match.string[end:]
+
+
+token_re = re.compile(r"^\s*([^=\s;,]+)")
+quoted_value_re = re.compile(r"^\s*=\s*\"([^\"\\]*(?:\\.[^\"\\]*)*)\"")
+value_re = re.compile(r"^\s*=\s*([^\s;,]*)")
+escape_re = re.compile(r"\\(.)")
+
+
+def split_header_words(header_values):
+ r"""Parse header values into a list of lists containing key,value pairs.
+
+ The function knows how to deal with ",", ";" and "=" as well as quoted
+ values after "=". A list of space separated tokens are parsed as if they
+ were separated by ";".
+
+ If the header_values passed as argument contains multiple values, then they
+ are treated as if they were a single value separated by comma ",".
+
+ This means that this function is useful for parsing header fields that
+ follow this syntax (BNF as from the HTTP/1.1 specification, but we relax
+ the requirement for tokens).
+
+ headers = #header
+ header = (token | parameter) *( [";"] (token | parameter))
+
+ token = 1*
+ separators = "(" | ")" | "<" | ">" | "@"
+ | "," | ";" | ":" | "\" | <">
+ | "/" | "[" | "]" | "?" | "="
+ | "{" | "}" | SP | HT
+
+ quoted-string = ( <"> *(qdtext | quoted-pair ) <"> )
+ qdtext = >
+ quoted-pair = "\" CHAR
+
+ parameter = attribute "=" value
+ attribute = token
+ value = token | quoted-string
+
+ Each header is represented by a list of key/value pairs. The value for a
+ simple token (not part of a parameter) is None. Syntactically incorrect
+ headers will not necessarily be parsed as you would want.
+
+ This is easier to describe with some examples:
+
+ >>> split_header_words(['foo="bar"; port="80,81"; discard, bar=baz'])
+ [[('foo', 'bar'), ('port', '80,81'), ('discard', None)], [('bar', 'baz')]]
+ >>> split_header_words(['text/html; charset="iso-8859-1"'])
+ [[('text/html', None), ('charset', 'iso-8859-1')]]
+ >>> split_header_words([r'Basic realm="\"foo\bar\""'])
+ [[('Basic', None), ('realm', '"foobar"')]]
+
+ """
+ assert not is_string(header_values)
+ result = []
+ for text in header_values:
+ orig_text = text
+ pairs = []
+ while text:
+ m = token_re.search(text)
+ if m:
+ text = unmatched(m)
+ name = m.group(1)
+ m = quoted_value_re.search(text)
+ if m: # quoted value
+ text = unmatched(m)
+ value = m.group(1)
+ value = escape_re.sub(r"\1", value)
+ else:
+ m = value_re.search(text)
+ if m: # unquoted value
+ text = unmatched(m)
+ value = m.group(1)
+ value = value.rstrip()
+ else:
+ # no value, a lone token
+ value = None
+ pairs.append((name, value))
+ elif text.lstrip().startswith(","):
+ # concatenated headers, as per RFC 2616 section 4.2
+ text = text.lstrip()[1:]
+ if pairs:
+ result.append(pairs)
+ pairs = []
+ else:
+ # skip junk
+ non_junk, nr_junk_chars = re.subn(r"^[=\s;]*", "", text)
+ assert nr_junk_chars > 0, (
+ "split_header_words bug: '%s', '%s', %s" %
+ (orig_text, text, pairs))
+ text = non_junk
+ if pairs:
+ result.append(pairs)
+ return result
+
+
+join_escape_re = re.compile(r"([\"\\])")
+
+
+def join_header_words(lists):
+ """Do the inverse of the conversion done by split_header_words.
+
+ Takes a list of lists of (key, value) pairs and produces a single header
+ value. Attribute values are quoted if needed.
+
+ >>> join_header_words([[("text/plain", None), ("charset", "iso-8859/1")]])
+ 'text/plain; charset="iso-8859/1"'
+ >>> join_header_words([[(\
+ "text/plain", None)], [("charset", "iso-8859/1")]])
+ 'text/plain, charset="iso-8859/1"'
+
+ """
+ headers = []
+ for pairs in lists:
+ attr = []
+ for k, v in pairs:
+ if v is not None:
+ if not re.search(r"^\w+$", v):
+ v = join_escape_re.sub(r"\\\1", v) # escape " and \
+ v = '"%s"' % v
+ if k is None: # Netscape cookies may have no name
+ k = v
+ else:
+ k = "%s=%s" % (k, v)
+ attr.append(k)
+ if attr:
+ headers.append("; ".join(attr))
+ return ", ".join(headers)
+
+
+def strip_quotes(text):
+ if text.startswith('"'):
+ text = text[1:]
+ if text.endswith('"'):
+ text = text[:-1]
+ return text
+
+
+def parse_ns_headers(ns_headers):
+ """Ad-hoc parser for Netscape protocol cookie-attributes.
+
+ The old Netscape cookie format for Set-Cookie can for instance contain
+ an unquoted "," in the expires field, so we have to use this ad-hoc
+ parser instead of split_header_words.
+
+ XXX This may not make the best possible effort to parse all the crap
+ that Netscape Cookie headers contain. Ronald Tschalar's HTTPClient
+ parser is probably better, so could do worse than following that if
+ this ever gives any trouble.
+
+ Currently, this is also used for parsing RFC 2109 cookies.
+
+ """
+ known_attrs = (
+ "expires",
+ "domain",
+ "path",
+ "secure",
+ # RFC 2109 attrs (may turn up in Netscape cookies, too)
+ "version",
+ "port",
+ "max-age")
+
+ result = []
+ for ns_header in ns_headers:
+ pairs = []
+ version_set = False
+ params = re.split(r";\s*", ns_header)
+ for ii in range(len(params)):
+ param = params[ii]
+ param = param.rstrip()
+ if param == "":
+ continue
+ if "=" not in param:
+ k, v = param, None
+ else:
+ k, v = re.split(r"\s*=\s*", param, 1)
+ k = k.lstrip()
+ if ii != 0:
+ lc = k.lower()
+ if lc in known_attrs:
+ k = lc
+ if k == "version":
+ # This is an RFC 2109 cookie.
+ v = strip_quotes(v)
+ version_set = True
+ if k == "expires":
+ # convert expires date to seconds since epoch
+ v = http2time(strip_quotes(v)) # None if invalid
+ pairs.append((k, v))
+
+ if pairs:
+ if not version_set:
+ pairs.append(("version", "0"))
+ result.append(pairs)
+
+ return result
+
+
+uppercase_headers = {'WWW', 'TE'}
+
+
+def normalize_header_name(name):
+ parts = [x.capitalize() for x in name.split('-')]
+ q = parts[0].upper()
+ if q in uppercase_headers:
+ parts[0] = q
+ if len(parts) == 3 and parts[1] == 'Websocket':
+ parts[1] = 'WebSocket'
+ return '-'.join(parts)
+
+
+def _test():
+ import doctest
+ from . import _headersutil
+ return doctest.testmod(_headersutil)
+
+
+if __name__ == "__main__":
+ _test()
diff --git a/core/lib/mechanize/_html.py b/core/lib/mechanize/_html.py
new file mode 100644
index 00000000..a50a17f0
--- /dev/null
+++ b/core/lib/mechanize/_html.py
@@ -0,0 +1,338 @@
+from __future__ import absolute_import
+
+import codecs
+import copy
+import re
+
+from ._form import parse_forms
+from ._headersutil import is_html as _is_html
+from ._headersutil import split_header_words
+from ._rfc3986 import clean_url, urljoin
+from .polyglot import is_string
+
+DEFAULT_ENCODING = "utf-8"
+_encoding_pats = (
+ # XML declaration
+ r'<\?[^<>]+encoding\s*=\s*[\'"](.*?)[\'"][^<>]*>',
+ # HTML 5 charset
+ r''']*>(?:\s*){0,1}''',
+ # HTML 4 Pragma directive
+ r''']*?content\s*=\s*['"][^'"]*?charset=([-_a-z0-9]+)[^'"]*?['"][^<>]*>(?:\s*){0,1}''',
+)
+
+
+def compile_pats(binary):
+ for raw in _encoding_pats:
+ if binary:
+ raw = raw.encode('ascii')
+ yield re.compile(raw, flags=re.IGNORECASE)
+
+
+class LazyEncodingPats(object):
+
+ def __call__(self, binary=False):
+ attr = 'binary_pats' if binary else 'unicode_pats'
+ pats = getattr(self, attr, None)
+ if pats is None:
+ pats = tuple(compile_pats(binary))
+ setattr(self, attr, pats)
+ for pat in pats:
+ yield pat
+
+
+lazy_encoding_pats = LazyEncodingPats()
+
+
+def find_declared_encoding(raw, limit=50*1024):
+ prefix = raw[:limit]
+ is_binary = isinstance(raw, bytes)
+ for pat in lazy_encoding_pats(is_binary):
+ m = pat.search(prefix)
+ if m is not None:
+ ans = m.group(1)
+ if is_binary:
+ ans = ans.decode('ascii', 'replace')
+ return ans
+
+
+def elem_text(elem):
+ if elem.text:
+ yield elem.text
+ for child in elem:
+ for text in elem_text(child):
+ yield text
+ if child.tail:
+ yield child.tail
+
+
+def iterlinks(root, base_url):
+ link_tags = {"a": "href", "area": "href", "iframe": "src"}
+ for tag in root.iter('*'):
+ if not is_string(tag.tag):
+ continue
+ q = tag.tag.lower()
+ attr = link_tags.get(q)
+ if attr is not None:
+ val = tag.get(attr)
+ if val:
+ url = clean_url(http://github.com/val)
+ yield Link(base_url, url,
+ compress_whitespace(u''.join(elem_text(tag))), q,
+ tag.items())
+ elif q == 'base':
+ href = tag.get('href')
+ if href:
+ base_url = href
+
+
+def compress_whitespace(text):
+ return re.sub(r'\s+', ' ', text or '').strip()
+
+
+def get_encoding_from_response(response, verify=True):
+ # HTTPEquivProcessor may be in use, so both HTTP and HTTP-EQUIV
+ # headers may be in the response. HTTP-EQUIV headers come last,
+ # so try in order from first to last.
+ if response:
+ for ct in response.info().getheaders("content-type"):
+ for k, v in split_header_words([ct])[0]:
+ if k == "charset":
+ if not verify:
+ return v
+ try:
+ codecs.lookup(v)
+ return v
+ except LookupError:
+ continue
+
+
+class EncodingFinder:
+ def __init__(self, default_encoding):
+ self._default_encoding = default_encoding
+
+ def encoding(self, response):
+ return get_encoding_from_response(response) or self._default_encoding
+
+
+class ResponseTypeFinder:
+ def __init__(self, allow_xhtml):
+ self._allow_xhtml = allow_xhtml
+
+ def is_html(self, response, encoding):
+ ct_hdrs = response.info().getheaders("content-type")
+ url = response.geturl()
+ # XXX encoding
+ return _is_html(ct_hdrs, url, self._allow_xhtml)
+
+
+class Link:
+ '''
+ A link in a HTML document
+
+ :ivar absolute_url: The absolutized link URL
+ :ivar url: The link URL
+ :ivar base_url: The base URL against which this link is resolved
+ :ivar text: The link text
+ :ivar tag: The link tag name
+ :ivar attrs: The tag attributes
+
+ '''
+ def __init__(self, base_url, url, text, tag, attrs):
+ assert None not in [url, tag, attrs]
+ self.base_url = base_url
+ self.absolute_url = urljoin(base_url, url)
+ self.url, self.text, self.tag, self.attrs = url, text, tag, attrs
+ self.text = self.text
+
+ def __eq__(self, other):
+ try:
+ for name in "url", "text", "tag":
+ if getattr(self, name) != getattr(other, name):
+ return False
+ if dict(self.attrs) != dict(other.attrs):
+ return False
+ except AttributeError:
+ return False
+ return True
+
+ def __ne__(self, other):
+ return not self.__eq__(other)
+
+ def __repr__(self):
+ return "Link(base_url=%r, url=%r, text=%r, tag=%r, attrs=%r)" % (
+ self.base_url, self.url, self.text, self.tag, self.attrs)
+
+
+def content_parser(data,
+ url=None,
+ response_info=None,
+ transport_encoding=None,
+ default_encoding=DEFAULT_ENCODING,
+ is_html=True):
+ '''
+ Parse data (a bytes object) into an etree representation such as
+ :py:mod:`xml.etree.ElementTree` or `lxml.etree`
+
+ :param bytes data: The data to parse
+ :param url: The URL of the document being parsed or None
+ :param response_info: Information about the document
+ (contains all HTTP headers as :class:`HTTPMessage`)
+ :param transport_encoding: The character encoding for the document being
+ parsed as specified in the HTTP headers or None.
+ :param default_encoding: The character encoding to use if no encoding
+ could be detected and no transport_encoding is specified
+ :param is_html: If the document is to be parsed as HTML.
+ '''
+ if not is_html:
+ return
+ try:
+ from html5_parser import parse
+ except Exception:
+ from html5lib import parse
+ kw = {'namespaceHTMLElements': False}
+ if transport_encoding and isinstance(data, bytes):
+ kw['transport_encoding'] = transport_encoding
+ return parse(data, **kw)
+ else:
+ return parse(data, transport_encoding=transport_encoding)
+
+
+def get_title(root):
+ for title in root.iter('title'):
+ text = compress_whitespace(title.text)
+ if text:
+ return text
+
+
+lazy = object()
+
+
+class Factory:
+ """Factory for forms, links, etc.
+
+ This interface may expand in future.
+
+ Public methods:
+
+ set_request_class(request_class)
+ set_response(response)
+ forms()
+ links()
+
+ Public attributes:
+
+ Note that accessing these attributes may raise ParseError.
+
+ encoding: string specifying the encoding of response if it contains a text
+ document (this value is left unspecified for documents that do not have
+ an encoding, e.g. an image file)
+ is_html: true if response contains an HTML document (XHTML may be
+ regarded as HTML too)
+ title: page title, or None if no title or not HTML
+ global_form: form object containing all controls that are not descendants
+ of any FORM element, or None if the forms_factory does not support
+ supplying a global form
+
+ """
+
+ def __init__(
+ self,
+ default_encoding=DEFAULT_ENCODING,
+ allow_xhtml=False, ):
+ """
+
+ Pass keyword arguments only.
+
+ """
+ self._encoding_finder = EncodingFinder(default_encoding)
+ self.form_encoding = default_encoding
+ self._response_type_finder = ResponseTypeFinder(
+ allow_xhtml=allow_xhtml)
+ self._content_parser = content_parser
+ self._current_forms = self._current_links = self._current_title = lazy
+ self._current_global_form = self._root = lazy
+ self._raw_data = b''
+ self.is_html, self.encoding = False, DEFAULT_ENCODING
+
+ self.set_response(None)
+
+ def set_content_parser(self, val):
+ self._content_parser = val
+
+ def set_request_class(self, request_class):
+ """Set request class (mechanize.Request by default).
+
+ HTMLForm instances returned by .forms() will return instances of this
+ class when .click()ed.
+
+ """
+ self._request_class = request_class
+
+ def set_response(self, response):
+ """Set response.
+
+ The response must either be None or implement the same interface as
+ objects returned by mechanize.urlopen().
+
+ """
+ self._response = copy.copy(response)
+ self._current_forms = self._current_links = self._current_title = lazy
+ self._current_global_form = self._root = lazy
+ self.encoding = self._encoding_finder.encoding(self._response)
+ self.is_html = self._response_type_finder.is_html(
+ self._response, self.encoding) if self._response else False
+
+ @property
+ def root(self):
+ if self._root is lazy:
+ response = self._response
+ raw = self._response.read() if self._response else b''
+ default_encoding = self._encoding_finder._default_encoding
+ transport_encoding = get_encoding_from_response(response, verify=False)
+ declared_encoding = find_declared_encoding(raw)
+ self.form_encoding = declared_encoding or transport_encoding or default_encoding
+ self._root = self._content_parser(
+ raw,
+ url=response.geturl() if response else None,
+ response_info=response.info() if response else None,
+ default_encoding=default_encoding,
+ is_html=self.is_html,
+ transport_encoding=transport_encoding)
+ return self._root
+
+ @property
+ def title(self):
+ if self._current_title is lazy:
+ self._current_title = get_title(
+ self.root) if self.root is not None else None
+ return self._current_title or u''
+
+ @property
+ def global_form(self):
+ if self._current_global_form is lazy:
+ self.forms()
+ return self._current_global_form
+
+ def forms(self):
+ """ Return tuple of HTMLForm-like objects. """
+ # this implementation sets .global_form as a side-effect
+ if self._current_forms is lazy:
+ self._current_forms, self._current_global_form = self._get_forms()
+ return self._current_forms
+
+ def links(self):
+ """Return tuple of mechanize.Link-like objects. """
+ if self._current_links is lazy:
+ self._current_links = self._get_links()
+ return self._get_links()
+
+ def _get_links(self):
+ if self.root is None:
+ return ()
+ return tuple(iterlinks(self.root, self._response.geturl()))
+
+ def _get_forms(self):
+ if self.root is None:
+ return (), None
+ return parse_forms(self.root,
+ self._response.geturl(), self._request_class, encoding=self.form_encoding)
diff --git a/core/lib/mechanize/_http.py b/core/lib/mechanize/_http.py
new file mode 100644
index 00000000..d6f8a55c
--- /dev/null
+++ b/core/lib/mechanize/_http.py
@@ -0,0 +1,312 @@
+"""HTTP related handlers.
+
+Note that some other HTTP handlers live in more specific modules: _auth.py,
+_gzip.py, etc.
+
+
+Copyright 2002-2006 John J Lee
+
+This code is free software; you can redistribute it and/or modify it
+under the terms of the BSD or ZPL 2.1 licenses (see the file
+LICENSE included with the distribution).
+
+"""
+
+from __future__ import absolute_import
+
+import logging
+import socket
+import time
+from io import BytesIO
+
+from . import _rfc3986, _sockettimeout
+from ._headersutil import is_html
+from ._request import Request
+from ._response import response_seek_wrapper
+from ._urllib2_fork import BaseHandler, HTTPError
+from ._equiv import HTTPEquivParser
+from .polyglot import create_response_info, RobotFileParser, is_py2, as_unicode
+
+debug = logging.getLogger("mechanize").debug
+debug_robots = logging.getLogger("mechanize.robots").debug
+
+
+def parse_head(fileobj):
+ """Return a list of key, value pairs."""
+ p = HTTPEquivParser(fileobj.read(4096))
+ return p()
+
+
+class HTTPEquivProcessor(BaseHandler):
+ """Append META HTTP-EQUIV headers to regular HTTP headers."""
+
+ handler_order = 300 # before handlers that look at HTTP headers
+
+ def http_response(self, request, response):
+ if not hasattr(response, "seek"):
+ response = response_seek_wrapper(response)
+ http_message = response.info()
+ url = response.geturl()
+ ct_hdrs = http_message.getheaders("content-type")
+ if is_html(ct_hdrs, url, True):
+ try:
+ try:
+ html_headers = parse_head(response)
+ finally:
+ response.seek(0)
+ except Exception:
+ pass
+ else:
+ for hdr, val in html_headers:
+ if is_py2:
+ # add a header
+ http_message.dict[hdr.lower()] = val
+ text = hdr + b": " + val
+ for line in text.split(b"\n"):
+ http_message.headers.append(line + b"\n")
+ else:
+ hdr = hdr.decode('iso-8859-1')
+ http_message[hdr] = val.decode('iso-8859-1')
+ return response
+
+ https_response = http_response
+
+
+class MechanizeRobotFileParser(RobotFileParser):
+
+ def __init__(self, url='', opener=None):
+ RobotFileParser.__init__(self, url)
+ self._opener = opener
+ self._timeout = _sockettimeout._GLOBAL_DEFAULT_TIMEOUT
+
+ def set_opener(self, opener=None):
+ from . import _opener
+ if opener is None:
+ opener = _opener.OpenerDirector()
+ self._opener = opener
+
+ def set_timeout(self, timeout):
+ self._timeout = timeout
+
+ def read(self):
+ """Reads the robots.txt URL and feeds it to the parser."""
+ if self._opener is None:
+ self.set_opener()
+ req = Request(self.url, unverifiable=True, visit=False,
+ timeout=self._timeout)
+ try:
+ f = self._opener.open(req)
+ except HTTPError as err:
+ f = err
+ except (IOError, socket.error, OSError) as exc:
+ debug_robots("ignoring error opening %r: %s" %
+ (self.url, exc))
+ return
+ lines = []
+ line = f.readline()
+ while line:
+ lines.append(line.strip())
+ line = f.readline()
+ status = f.code
+ if status == 401 or status == 403:
+ self.disallow_all = True
+ debug_robots("disallow all")
+ elif status >= 400:
+ self.allow_all = True
+ debug_robots("allow all")
+ elif status == 200 and lines:
+ debug_robots("parse lines")
+ if is_py2:
+ self.parse(lines)
+ else:
+ self.parse(map(as_unicode, lines))
+
+
+class RobotExclusionError(HTTPError):
+
+ def __init__(self, request, *args):
+ HTTPError.__init__(self, *args)
+ self.request = request
+
+
+class HTTPRobotRulesProcessor(BaseHandler):
+ # before redirections, after everything else
+ handler_order = 800
+ http_response_class = None
+
+ def __init__(self, rfp_class=MechanizeRobotFileParser):
+ self.rfp_class = rfp_class
+ self.rfp = None
+ self._host = None
+
+ def __copy__(self):
+ return self.__class__(self.rfp_class)
+
+ def http_request(self, request):
+ scheme = request.get_type()
+ if scheme not in ["http", "https"]:
+ # robots exclusion only applies to HTTP
+ return request
+
+ if request.get_selector() == "/robots.txt":
+ # /robots.txt is always OK to fetch
+ return request
+
+ host = request.get_host()
+
+ # robots.txt requests don't need to be allowed by robots.txt :-)
+ origin_req = getattr(request, "_origin_req", None)
+ if (origin_req is not None and
+ origin_req.get_selector() == "/robots.txt" and
+ origin_req.get_host() == host):
+ return request
+
+ if host != self._host:
+ self.rfp = self.rfp_class()
+ try:
+ self.rfp.set_opener(self.parent)
+ except AttributeError:
+ debug("%r instance does not support set_opener" %
+ self.rfp.__class__)
+ self.rfp.set_url(http://github.com/scheme%20+%20%22://%22%20+%20host%20+%20%22/robots.txt")
+ self.rfp.set_timeout(request.timeout)
+ self.rfp.read()
+ self._host = host
+
+ ua = request.get_header("User-agent", "")
+ if self.rfp.can_fetch(ua, request.get_full_url()):
+ return request
+ else:
+ # XXX This should really have raised URLError. Too late now...
+ factory = self.http_response_class or create_response_info
+ msg = b"request disallowed by robots.txt"
+ raise RobotExclusionError(
+ request,
+ request.get_full_url(),
+ 403, msg,
+ factory(BytesIO()), BytesIO(msg))
+
+ https_request = http_request
+
+
+class HTTPRefererProcessor(BaseHandler):
+ """Add Referer header to requests.
+
+ This only makes sense if you use each RefererProcessor for a single
+ chain of requests only (so, for example, if you use a single
+ HTTPRefererProcessor to fetch a series of URLs extracted from a single
+ page, this will break).
+
+ There's a proper implementation of this in mechanize.Browser.
+
+ """
+
+ def __init__(self):
+ self.referer = None
+
+ def http_request(self, request):
+ if ((self.referer is not None) and
+ not request.has_header("Referer")):
+ request.add_unredirected_header("Referer", self.referer)
+ return request
+
+ def http_response(self, request, response):
+ self.referer = response.geturl()
+ return response
+
+ https_request = http_request
+ https_response = http_response
+
+
+def clean_refresh_http://github.com/url(url):
+ # e.g. Firefox 1.5 does (something like) this
+ if ((url.startswith('"') and url.endswith('"')) or
+ (url.startswith("'") and url.endswith("'"))):
+ url = url[1:-1]
+ return _rfc3986.clean_url(http://github.com/url,%20'utf-8') # XXX encoding
+
+
+def parse_refresh_header(refresh):
+ """
+ >>> parse_refresh_header("1; url=http://example.com/")
+ (1.0, 'http://example.com/')
+ >>> parse_refresh_header("1; url='http://example.com/'")
+ (1.0, 'http://example.com/')
+ >>> parse_refresh_header("1")
+ (1.0, None)
+ >>> parse_refresh_header("blah") # doctest: +IGNORE_EXCEPTION_DETAIL
+ Traceback (most recent call last):
+ ValueError: invalid literal for float(): blah
+
+ """
+
+ ii = refresh.find(";")
+ if ii != -1:
+ pause, newurl_spec = float(refresh[:ii]), refresh[ii + 1:]
+ jj = newurl_spec.find("=")
+ key = None
+ if jj != -1:
+ key, newurl = newurl_spec[:jj], newurl_spec[jj + 1:]
+ newurl = clean_refresh_url(http://github.com/newurl)
+ if key is None or key.strip().lower() != "url":
+ raise ValueError()
+ else:
+ pause, newurl = float(refresh), None
+ return pause, newurl
+
+
+class HTTPRefreshProcessor(BaseHandler):
+ """Perform HTTP Refresh redirections.
+
+ Note that if a non-200 HTTP code has occurred (for example, a 30x
+ redirect), this processor will do nothing.
+
+ By default, only zero-time Refresh headers are redirected. Use the
+ max_time attribute / constructor argument to allow Refresh with longer
+ pauses. Use the honor_time attribute / constructor argument to control
+ whether the requested pause is honoured (with a time.sleep()) or
+ skipped in favour of immediate redirection.
+
+ Public attributes:
+
+ max_time: see above
+ honor_time: see above
+
+ """
+ handler_order = 1000
+
+ def __init__(self, max_time=0, honor_time=True):
+ self.max_time = max_time
+ self.honor_time = honor_time
+ self._sleep = time.sleep
+
+ def __copy__(self):
+ return self.__class__(self.max_time, self.honor_time)
+
+ def http_response(self, request, response):
+ code, msg, hdrs = response.code, response.msg, response.info()
+
+ if code == 200 and 'refresh' in hdrs:
+ refresh = hdrs.getheaders("refresh")[0]
+ try:
+ pause, newurl = parse_refresh_header(refresh)
+ except ValueError:
+ debug("bad Refresh header: %r" % refresh)
+ return response
+
+ if newurl is None:
+ newurl = response.geturl()
+ if (self.max_time is None) or (pause <= self.max_time):
+ if pause > 1E-3 and self.honor_time:
+ self._sleep(pause)
+ hdrs["location"] = newurl
+ # hardcoded http is NOT a bug
+ response = self.parent.error(
+ "http", request, response,
+ "refresh", msg, hdrs)
+ else:
+ debug("Refresh header ignored: %r" % refresh)
+
+ return response
+
+ https_response = http_response
diff --git a/core/lib/mechanize/_mechanize.py b/core/lib/mechanize/_mechanize.py
new file mode 100644
index 00000000..245d7957
--- /dev/null
+++ b/core/lib/mechanize/_mechanize.py
@@ -0,0 +1,840 @@
+"""Stateful programmatic WWW navigation, after Perl's WWW::Mechanize.
+
+Copyright 2003-2006 John J. Lee
+Copyright 2003 Andy Lester (original Perl code)
+
+This code is free software; you can redistribute it and/or modify it
+under the terms of the BSD or ZPL 2.1 licenses (see the file LICENSE
+included with the distribution).
+
+"""
+from __future__ import absolute_import
+
+import copy
+import os
+import re
+
+from . import _request, _response, _rfc3986, _sockettimeout, _urllib2_fork
+from ._clientcookie import Cookie
+from ._headersutil import normalize_header_name
+from ._html import Factory
+from ._useragent import UserAgentBase
+from .polyglot import pathname2url, HTTPError, is_string, iteritems
+
+
+class BrowserStateError(Exception):
+ pass
+
+
+class LinkNotFoundError(Exception):
+ pass
+
+
+class FormNotFoundError(Exception):
+ pass
+
+
+def sanepathname2url(http://github.com/path):
+ urlpath = pathname2url(http://github.com/path)
+ if os.name == "nt" and urlpath.startswith("///"):
+ urlpath = urlpath[2:]
+ # XXX don't ask me about the mac...
+ return urlpath
+
+
+class History:
+ """
+
+ Though this will become public, the implied interface is not yet stable.
+
+ """
+
+ def __init__(self):
+ self._history = [] # LIFO
+
+ def add(self, request, response):
+ self._history.append((request, response))
+
+ def back(self, n, _response):
+ response = _response # XXX move Browser._response into this class?
+ while n > 0 or response is None:
+ try:
+ request, response = self._history.pop()
+ except IndexError:
+ raise BrowserStateError("already at start of history")
+ n -= 1
+ return request, response
+
+ def clear(self):
+ del self._history[:]
+
+ def close(self):
+ for request, response in self._history:
+ if response is not None:
+ response.close()
+ del self._history[:]
+
+ def __copy__(self):
+ ans = self.__class__()
+ ans._history = self._history[:]
+ return ans
+
+
+class HTTPRefererProcessor(_urllib2_fork.BaseHandler):
+ def http_request(self, request):
+ # See RFC 2616 14.36. The only times we know the source of the
+ # request URI has a URI associated with it are redirect, and
+ # Browser.click() / Browser.submit() / Browser.follow_link().
+ # Otherwise, it's the user's job to add any Referer header before
+ # .open()ing.
+ if hasattr(request, "redirect_dict"):
+ request = self.parent._add_referer_header(
+ request, origin_request=False)
+ return request
+
+ https_request = http_request
+
+
+class Browser(UserAgentBase):
+ """Browser-like class with support for history, forms and links.
+
+ :class:`BrowserStateError` is raised whenever the browser is in the wrong
+ state to complete the requested operation - e.g., when :meth:`back()` is
+ called when the browser history is empty, or when :meth:`follow_link()` is
+ called when the current response does not contain HTML data.
+
+ Public attributes:
+
+ request: current request (:class:`mechanize.Request`)
+
+ form: currently selected form (see :meth:`select_form()`)
+
+ :param history: object implementing the :class:`mechanize.History`
+ interface. Note this interface is still experimental
+ and may change in future. This object is owned
+ by the browser instance and must not be shared
+ among browsers.
+ :param request_class: Request class to use. Defaults to
+ :class:`mechanize.Request`
+ :param content_parser: A function that is responsible for parsing
+ received html/xhtml content. See the builtin
+ :func:`mechanize._html.content_parser()` function for details
+ on the interface this function must support.
+
+ """
+
+ handler_classes = copy.copy(UserAgentBase.handler_classes)
+ handler_classes["_referer"] = HTTPRefererProcessor
+ default_features = copy.copy(UserAgentBase.default_features)
+ default_features.append("_referer")
+
+ def __init__(
+ self,
+ history=None,
+ request_class=None,
+ content_parser=None,
+ allow_xhtml=False, ):
+ """
+ Only named arguments should be passed to this constructor.
+
+ """
+ self._handle_referer = True
+
+ if history is None:
+ history = History()
+ self._history = history
+
+ if request_class is None:
+ request_class = _request.Request
+
+ factory = Factory(allow_xhtml=allow_xhtml)
+ factory.set_request_class(request_class)
+ if content_parser is not None:
+ factory.set_content_parser(content_parser)
+ self._factory = factory
+ self.request_class = request_class
+
+ self.request = None
+ self._set_response(None, False)
+
+ # do this last to avoid __getattr__ problems
+ UserAgentBase.__init__(self)
+
+ def __copy__(self):
+ '''
+ Clone this browser instance. The clone will share the same, thread-safe
+ cookie jar, and have all the same handlers/settings, but will not share
+ any other state, making it safe to use in another thread.
+ '''
+ ans = self.__class__()
+ self._copy_state(ans)
+ ans._handle_referer = self._handle_referer
+ for attr in ('_response_type_finder', '_encoding_finder',
+ '_content_parser'):
+ setattr(ans._factory, attr, getattr(self._factory, attr))
+ ans.request_class = self.request_class
+ ans._history = copy.copy(self._history)
+ return ans
+
+ def close(self):
+ UserAgentBase.close(self)
+ if self._response is not None:
+ self._response.close()
+ if self._history is not None:
+ self._history.close()
+ self._history = None
+
+ # make use after .close easy to spot
+ self.form = None
+ self.request = self._response = None
+ self.request = self.response = self.set_response = None
+ self.geturl = self.reload = self.back = None
+ self.clear_history = self.set_cookie = self.links = self.forms = None
+ self.viewing_html = self.encoding = self.title = None
+ self.select_form = self.click = self.submit = self.click_link = None
+ self.follow_link = self.find_link = None
+
+ def set_handle_referer(self, handle):
+ """Set whether to add Referer header to each request."""
+ self._set_handler("_referer", handle)
+ self._handle_referer = bool(handle)
+
+ def _add_referer_header(self, request, origin_request=True):
+ if self.request is None:
+ return request
+ scheme = request.get_type()
+ original_scheme = self.request.get_type()
+ if scheme not in ["http", "https"]:
+ return request
+ if not origin_request and not self.request.has_header("Referer"):
+ return request
+
+ if (self._handle_referer and original_scheme in ["http", "https"] and
+ not (original_scheme == "https" and scheme != "https")):
+ # strip URL fragment (RFC 2616 14.36)
+ parts = _rfc3986.urlsplit(self.request.get_full_url())
+ parts = parts[:-1] + (None, )
+ referer = _rfc3986.urlunsplit(parts)
+ request.add_unredirected_header("Referer", referer)
+ return request
+
+ def open_novisit(self,
+ url_or_request,
+ data=None,
+ timeout=_sockettimeout._GLOBAL_DEFAULT_TIMEOUT):
+ """Open a URL without visiting it.
+
+ Browser state (including request, response, history, forms and links)
+ is left unchanged by calling this function.
+
+ The interface is the same as for :meth:`open()`.
+
+ This is useful for things like fetching images.
+
+ See also :meth:`retrieve()`
+
+ """
+ return self._mech_open(
+ url_or_request, data, visit=False, timeout=timeout)
+
+ def open(self,
+ url_or_request,
+ data=None,
+ timeout=_sockettimeout._GLOBAL_DEFAULT_TIMEOUT):
+ '''
+ Open a URL. Loads the page so that you can subsequently use
+ :meth:`forms()`, :meth:`links()`, etc. on it.
+
+ :param url_or_request: Either a URL or a :class:`mechanize.Request`
+ :param dict data: data to send with a POST request
+ :param timeout: Timeout in seconds
+ :return: A :class:`mechanize.Response` object
+ '''
+ return self._mech_open(url_or_request, data, timeout=timeout)
+
+ def _mech_open(self,
+ url,
+ data=None,
+ update_history=True,
+ visit=None,
+ timeout=_sockettimeout._GLOBAL_DEFAULT_TIMEOUT):
+ try:
+ url.get_full_url
+ except AttributeError:
+ # string URL -- convert to absolute URL if required
+ scheme, authority = _rfc3986.urlsplit(url)[:2]
+ if scheme is None:
+ # relative URL
+ if self._response is None:
+ raise BrowserStateError("can't fetch relative reference: "
+ "not viewing any document")
+ url = _rfc3986.urljoin(self._response.geturl(), url)
+
+ request = self._request(url, data, visit, timeout)
+ visit = request.visit
+ if visit is None:
+ visit = True
+
+ if visit:
+ self._visit_request(request, update_history)
+
+ success = True
+ try:
+ response = UserAgentBase.open(self, request, data)
+ except HTTPError as error:
+ success = False
+ if error.fp is None: # not a response
+ raise
+ response = error
+
+# except (IOError, socket.error, OSError) as error:
+# Yes, urllib2 really does raise all these :-((
+# See test_urllib2.py for examples of socket.gaierror and OSError,
+# plus note that FTPHandler raises IOError.
+# XXX I don't seem to have an example of exactly socket.error being
+# raised, only socket.gaierror...
+# I don't want to start fixing these here, though, since this is a
+# subclass of OpenerDirector, and it would break old code. Even in
+# Python core, a fix would need some backwards-compat. hack to be
+# acceptable.
+# raise
+
+ if visit:
+ self._set_response(response, False)
+ response = copy.copy(self._response)
+ elif response is not None:
+ response = _response.upgrade_response(response)
+
+ if not success:
+ raise response
+ return response
+
+ def __str__(self):
+ text = []
+ text.append("<%s " % self.__class__.__name__)
+ if self._response:
+ text.append("visiting %s" % self._response.geturl())
+ else:
+ text.append("(not visiting a URL)")
+ if self.form:
+ text.append("\n selected form:\n %s\n" % str(self.form))
+ text.append(">")
+ return "".join(text)
+
+ def response(self):
+ """Return a copy of the current response.
+
+ The returned object has the same interface as the object returned by
+ :meth:`.open()`
+
+ """
+ return copy.copy(self._response)
+
+ def open_local_file(self, filename):
+ path = sanepathname2url(http://github.com/os.path.abspath(filename))
+ url = 'file://' + path
+ return self.open(url)
+
+ def set_response(self, response):
+ """Replace current response with (a copy of) response.
+
+ response may be None.
+
+ This is intended mostly for HTML-preprocessing.
+ """
+ self._set_response(response, True)
+
+ def _set_response(self, response, close_current):
+ # sanity check, necessary but far from sufficient
+ if not (response is None or
+ (hasattr(response, "info") and hasattr(response, "geturl") and
+ hasattr(response, "read"))):
+ raise ValueError("not a response object")
+
+ self.form = None
+ if response is not None:
+ response = _response.upgrade_response(response)
+ if close_current and self._response is not None:
+ self._response.close()
+ self._response = response
+ self._factory.set_response(response)
+
+ def visit_response(self, response, request=None):
+ """Visit the response, as if it had been :meth:`open()` ed.
+
+ Unlike :meth:`set_response()`, this updates history rather than
+ replacing the current response.
+ """
+ if request is None:
+ request = _request.Request(response.geturl())
+ self._visit_request(request, True)
+ self._set_response(response, False)
+
+ def _visit_request(self, request, update_history):
+ if self._response is not None:
+ self._response.close()
+ if self.request is not None and update_history:
+ self._history.add(self.request, self._response)
+ self._response = None
+ # we want self.request to be assigned even if UserAgentBase.open
+ # fails
+ self.request = request
+
+ def geturl(http://github.com/self):
+ """Get URL of current document."""
+ if self._response is None:
+ raise BrowserStateError("not viewing any document")
+ return self._response.geturl()
+
+ def reload(self):
+ """Reload current document, and return response object."""
+ if self.request is None:
+ raise BrowserStateError("no URL has yet been .open()ed")
+ if self._response is not None:
+ self._response.close()
+ return self._mech_open(self.request, update_history=False)
+
+ def back(self, n=1):
+ """Go back n steps in history, and return response object.
+
+ n: go back this number of steps (default 1 step)
+
+ """
+ if self._response is not None:
+ self._response.close()
+ self.request, response = self._history.back(n, self._response)
+ self.set_response(response)
+ if not response.read_complete:
+ return self.reload()
+ return copy.copy(response)
+
+ def clear_history(self):
+ self._history.clear()
+
+ def set_cookie(self, cookie_string):
+ """Set a cookie.
+
+ Note that it is NOT necessary to call this method under ordinary
+ circumstances: cookie handling is normally entirely automatic. The
+ intended use case is rather to simulate the setting of a cookie by
+ client script in a web page (e.g. JavaScript). In that case, use of
+ this method is necessary because mechanize currently does not support
+ JavaScript, VBScript, etc.
+
+ The cookie is added in the same way as if it had arrived with the
+ current response, as a result of the current request. This means that,
+ for example, if it is not appropriate to set the cookie based on the
+ current request, no cookie will be set.
+
+ The cookie will be returned automatically with subsequent responses
+ made by the Browser instance whenever that's appropriate.
+
+ cookie_string should be a valid value of the Set-Cookie header.
+
+ For example:
+
+ .. code-block:: python
+
+ browser.set_cookie(
+ "sid=abcdef; expires=Wednesday, 09-Nov-06 23:12:40 GMT")
+
+ Currently, this method does not allow for adding RFC 2986 cookies.
+ This limitation will be lifted if anybody requests it.
+
+ See also :meth:`set_simple_cookie()` for an easier way to set cookies
+ without needing to create a Set-Cookie header string.
+ """
+ if self._response is None:
+ raise BrowserStateError("not viewing any document")
+ if self.request.get_type() not in ["http", "https"]:
+ raise BrowserStateError("can't set cookie for non-HTTP/HTTPS "
+ "transactions")
+ cookiejar = self._ua_handlers["_cookies"].cookiejar
+ response = self.response() # copy
+ headers = response.info()
+ headers["Set-cookie"] = cookie_string
+ cookiejar.extract_cookies(response, self.request)
+
+ def set_simple_cookie(self, name, value, domain, path='/'):
+ '''
+ Similar to :meth:`set_cookie()` except that instead of using a
+ cookie string, you simply specify the `name`, `value`, `domain`
+ and optionally the `path`.
+ The created cookie will never expire. For example:
+
+ .. code-block:: python
+
+ browser.set_simple_cookie('some_key', 'some_value', '.example.com',
+ path='/some-page')
+ '''
+ self.cookiejar.set_cookie(
+ Cookie(0, name, value, None, False, domain, True, False, path,
+ True, False, None, False, None, None, None))
+
+ @property
+ def cookiejar(self):
+ ' Return the current cookiejar (:class:`mechanize.CookieJar`) or None '
+ try:
+ return self._ua_handlers["_cookies"].cookiejar
+ except Exception:
+ pass
+
+ def set_header(self, header, value=None):
+ '''
+ Convenience method to set a header value in `self.addheaders`
+ so that the header is sent out with all requests automatically.
+
+ :param header: The header name, e.g. User-Agent
+ :param value: The header value. If set to None the header is removed.
+ '''
+ found = False
+ header = normalize_header_name(header)
+ q = header.lower()
+ remove = []
+ for i, (k, v) in enumerate(tuple(self.addheaders)):
+ if k.lower() == q:
+ if value:
+ self.addheaders[i] = (header, value)
+ found = True
+ else:
+ remove.append(i)
+ if not found:
+ self.addheaders.append((header, value))
+ if remove:
+ for i in reversed(remove):
+ del self.addheaders[i]
+
+ def links(self, **kwds):
+ """Return iterable over links (:class:`mechanize.Link` objects)."""
+ if not self.viewing_html():
+ raise BrowserStateError("not viewing HTML")
+ links = self._factory.links()
+ if kwds:
+ return self._filter_links(links, **kwds)
+ else:
+ return links
+
+ def forms(self):
+ """Return iterable over forms.
+
+ The returned form objects implement the :class:`mechanize.HTMLForm`
+ interface.
+
+ """
+ if not self.viewing_html():
+ raise BrowserStateError("not viewing HTML")
+ return self._factory.forms()
+
+ def global_form(self):
+ """Return the global form object, or None if the factory implementation
+ did not supply one.
+
+ The "global" form object contains all controls that are not descendants
+ of any FORM element.
+
+ The returned form object implements the :class:`mechanize.HTMLForm`
+ interface.
+
+ This is a separate method since the global form is not regarded as part
+ of the sequence of forms in the document -- mostly for
+ backwards-compatibility.
+
+ """
+ if not self.viewing_html():
+ raise BrowserStateError("not viewing HTML")
+ return self._factory.global_form
+
+ def viewing_html(self):
+ """Return whether the current response contains HTML data."""
+ if self._response is None:
+ raise BrowserStateError("not viewing any document")
+ return self._factory.is_html
+
+ def encoding(self):
+ if self._response is None:
+ raise BrowserStateError("not viewing any document")
+ return self._factory.encoding
+
+ def title(self):
+ ' Return title, or None if there is no title element in the document. '
+ if not self.viewing_html():
+ raise BrowserStateError("not viewing HTML")
+ return self._factory.title
+
+ def select_form(self, name=None, predicate=None, nr=None, **attrs):
+ """Select an HTML form for input.
+
+ This is a bit like giving a form the "input focus" in a browser.
+
+ If a form is selected, the Browser object supports the HTMLForm
+ interface, so you can call methods like :meth:`set_value()`,
+ :meth:`set()`, and :meth:`click()`.
+
+ Another way to select a form is to assign to the .form attribute. The
+ form assigned should be one of the objects returned by the
+ :meth:`forms()` method.
+
+ If no matching form is found,
+ :class:`mechanize.FormNotFoundError` is raised.
+
+ If `name` is specified, then the form must have the indicated name.
+
+ If `predicate` is specified, then the form must match that function.
+ The predicate function is passed the :class:`mechanize.HTMLForm` as its
+ single argument, and should return a boolean value indicating whether
+ the form matched.
+
+ `nr`, if supplied, is the sequence number of the form (where 0 is the
+ first). Note that control 0 is the first form matching all the other
+ arguments (if supplied); it is not necessarily the first control in the
+ form. The "global form" (consisting of all form controls not contained
+ in any FORM element) is considered not to be part of this sequence and
+ to have no name, so will not be matched unless both name and nr are
+ None.
+
+ You can also match on any HTML attribute of the ` 
- 
-
-
- 
-
-
- 
-
-
- 
-
-
- 
-
-
- 
-
-
- 
-
-
- 
+
+ 
+
+ 
+ 
+- A complete versatile framework to cover up everything from Reconnaissance to Vulnerability Analysis.
+- Has 5 main phases, subdivided into __14 sub-phases__ consisting a total of __108 modules__.
+- Reconnaissance Phase has 50 modules of its own (including active and passive recon, information disclosure modules).
+- Scanning & Enumeration Phase has got 16 modules (including port scans, WAF analysis, etc)
+- Vulnerability Analysis Phase has 37 modules (including most common vulnerabilities in action).
+- Exploits Castle has only 1 exploit. `(purely developmental)`
+- And finally, Auxiliaries have got 4 modules. `more under development`
+- All four phases each have an `Auto-Awesome` module which automates every module for you.
+- huge performance boost through multiprocessing
+- Piping Attacks through Tor (not implemented everywhere yet)
+- You just need the domain, and leave everything is to this tool.
+- TIDoS has full verbose out support, so you'll know whats going on.
+- Attacking now even easier with a new GUI
+
+### Main new features
+- the programming language: TIDoS is fully ported to Python3
+- the interface: TIDoS presents a new, Metasploit-like console interface
+- Parallelisation: TIDoS uses multiprocessing to speed up attacks
+- An alternative CLI interface for faster interaction with one specific module
+- Anonymity: Attacking through Tor is possible (95% done)
+- Module Completion: Some modules have been feature-extended (e.g. more evasion, supporting more than 1 query parameter)
+- Some new modules: arpscan
+- A Graphical User Interface for easier interaction with the toolkit
+- Supports non-default http(s) ports
+
+### Upcoming
+- results of modules will be stored in a database
+- new modules: nikto&photon
### Installation :-
-#### Global Installation:
+#### Installation Script (Globally) :
-> NOTE: For installing globally, you will need to default your Python version to 2.x.
+To install the framework globally in /opt, run the provided `core/install.py` script as root. After this, you can launch TIDoS simply by typing `tidos` on the command line.
+
+#### Manual Installation (Locally) :
* Clone the repository locally and navigate there:
```
-git clone https://github.com/theinfecteddrake/tidos-framework.git
+git clone https://github.com/0xinfection/tidos-framework.git
cd tidos-framework
```
-* Install the dependencies:
+
+TIDoS needs some libraries to run, which can be installed via `aptitude` or `dnf` Package Managers.
+```
+sudo apt-get install libncurses5 libxml2 nmap tcpdump libexiv2-dev build-essential python3-pip libmariadbclient18 libmysqlclient-dev tor konsole
+```
+Now after these dependencies are finished installing, we need to install the remaining Python Package dependencies, hence run:
```
-chmod +x install
-./install
+pip3 install -r requirements.txt
```
-
-Thats it! Now you are good to go! Now lets run the tool:
+TIDoS uses Vailyn to scan for path traversals in a new, improved path traversal module. If you want to use that module, head to `https://github.com/VainlyStrain/Vailyn`, and follow the installation instructions there.
+
+Thats it. You now have TIDoS at your service. Fire it up using:
```
-tidos
+python3 tidv2 #Qt5 interface
+sudo python3 tidconsole.py #console interface
```
-#### Manual Installation (Locally) :
+#### Docker image :
-TIDoS needs some libraries to run, which can be installed via `aptitude` or `yum` Package Managers.
+You can build it from Dockerfile :
```
-sudo apt-get install libncurses5 libxml2 nmap tcpdump scons libexiv2-dev python-pyexiv2 build-essential python-pip default-libmysqlclient-dev python-xmpp
+git clone https://github.com/0xinfection/tidos-framework.git
+cd tidos-framework/core/docker
+docker build -t tidos .
```
-Now after these dependencies are finished installing, we need to install the remaining Python Package dependencies, hence run:
+
+To run TIDoS :
+
```
-pip2 install -r requirements.txt
+docker run --interactive --tty --rm tidos bash
+tidos
```
-Thats it, You now have TIDoS at your service. Fire it up using:
+
+Update: TIDoS is now available on Docker Hub. Install and run the container like this:
+
```
-python2 tidos.py
+docker run -it vainlystrain/tidos-framework
```
-### Getting Started :-
+#### Updating TIDoS :
-TIDoS is built to be a comprehensive, flexible and versatile framework where you just have to select and use modules.
+To get the current version of TIDoS, move into the installation folder and perform `(sudo) git pull #sudo if installed by install.py`. Alternatively, you can run the `fetch` command in tidconsole.
-So to get started, you need to set your own `API KEYS` for various OSINT & Scanning and Enumeration purposes. To do so, open up `API_KEYS.py` under `files/` directory and set your own keys and access tokens for `SHODAN`, `CENSYS`, `FULL CONTACT`, `GOOGLE` and `WHATCMS`. Public `API KEYS` and `ACCESS TOKENS` for `SHODAN` and `WHATCMS` have been provided with the TIDoS release itself. You can still add your own... `no harm!`
+### Getting Started :-
-Finally, as the framework opens up, enter the website name `eg. http://www.example.com` and let TIDoS lead you. Thats it! Its as easy as that.
+To get started, you need to set your own `API KEYS` for various OSINT & Scanning and Enumeration purposes. To do so, open up `API_KEYS.py` under `files/` directory and set your own keys and access tokens for `SHODAN`, `CENSYS`, `FULL CONTACT`, `GOOGLE` and `WHATCMS`.
-> Recommended:
-> - Follow the order of the tool (Run in a schematic way).
+> __GOOD NEWS__:
>
-> `Reconnaissance ➣ Scanning & Enumeration ➣ Vulnerability Analysis`
+> The latest release of TIDoS includes all API KEYS and ACCESS TOKENS for `SHODAN`, `CENSYS`, `FULL CONTACT`, `GOOGLE` and `WHATCMS` by default. I found these tokens on various repositories on GitHub itself. __You can now use all the modules__ which use the API KEYS. :)
-To update this tool, use `tidos_updater.py` module under `tools/` folder.
+#### Commands :-
+
+```
+__ __
+ ! attack Attack specified target(s) M
+ : clear Clear terminal. :
+ V creds Handle target credentials.
+ : fetch Check for and install updates. :
+ : find Search a module. :
+ help Show help message. :
+ info Show description of current module. M
+ : intro Display Intro. :
+ : leave Leave module. M
+ list List all modules of a category. :
+ : load Load module. :
+ : netinfo Show network information. :
+ : opts Show options of current module. M
+ phpsploit Load the phpsploit framework. :
+ (needs to be downloaded externally)
+ : processes Set number of processes in parallelis. :
+ q Terminate TIDoS session. :
+ : sessions Interact with cached sessions. :
+ : set Set option value of module. M
+ : tor Pipe Attacks through the Tor Network. :
+ vicadd Add Target to list. :
+ vicdel Delete Target from list. :
+ viclist List all targets. :
+
+ Avail. Cmds
+ M needs loaded modvle
+ V [! potentially] need loaded target(s)
+```
### Flawless Features :-
-TIDoS Framework presently supports the following: `and more modules are under active development`
+TIDoS presently supports the following: `and more modules are under active development`
* __Reconnaissance + OSINT__
- + Passive Reconnaissance:
+ + __Passive Reconnaissance:__
- Nping Enumeration `Via external APi`
- WhoIS Lookup `Domain info gathering`
- GeoIP Lookup `Pinpoint physical location`
@@ -136,7 +178,7 @@ TIDoS Framework presently supports the following: `and more modules are under ac
- Censys Intel Gathering `Domain Based`
- Threat Intelligence Gathering `Bad IPs`
- + Active Reconnaissance
+ + __Active Reconnaissance:__
- Ping Enumeration `Advanced`
- CMS Detection `(185+ CMSs supported)` `IMPROVED`
- Advanced Traceroute `IMPROVED`
@@ -160,7 +202,7 @@ TIDoS Framework presently supports the following: `and more modules are under ac
- Common Dot Files `.htaccess, .apache, etc`
- Common Logfile Locations `.log, .changelog, etc`
- + Information Disclosure
+ + __Information Disclosure:__
- Credit Cards Disclosure `If Plaintext`
- Email Harvester `IMPROVED`
- Fatal Errors Enumeration `Includes Full Path Disclosure`
@@ -188,8 +230,9 @@ TIDoS Framework presently supports the following: `and more modules are under ac
- Depth 1 `Indexed Uri Crawler`
- Depth 2 `Single Page Crawler`
- Depth 3 `Web Link Crawler`
+ + ARP Scanner `NEW`
-+ __Vulnerability Analysis__
+* __Vulnerability Analysis__
__Web-Bugs & Server Misconfigurations__
@@ -227,7 +270,7 @@ TIDoS Framework presently supports the following: `and more modules are under ac
- Parameter Based
- Pre-loaded Path Based
+ OS Command Injection `Linux & Windows (RCE)`
- + Path Traversal `(Sensitive Paths)`
+ + Path Traversal `ENHANCED`
+ Cross-Site Request Forgery `Absolute`
+ SQL Injection
+ Error Based Injection
@@ -267,7 +310,7 @@ TIDoS Framework presently supports the following: `and more modules are under ac
- SSH Protocol Bruteforce
- POP 2/3 Protocol Bruteforce
- SQL Protocol Bruteforce
- - XMPP Protocol Bruteforce
+ - (XMPP Protocol Bruteforce) `BROKEN:DEP`
- SMTP Protocol Bruteforce
- TELNET Protocol Bruteforce
@@ -283,63 +326,22 @@ TIDoS Framework presently supports the following: `and more modules are under ac
+ ShellShock
### Other Tools:
-- `net_info.py` - Displays information about your network. Located under `tools/`.
-- `tidos_updater.py` - Updates the framework to the latest release via signature matching. Located under `tools/`.
+- `net_info.py` - Displays information about your network. Accessible from 'netinfo' command.
### TIDoS In Action:
-Lets see some screenshots of TIDoS in real world pentesting action:
-
-
-
-
-
-
+Lets see a demonstration of TIDoS in action:
-
-
-
+[](https://asciinema.org/a/359477)
### Version:
```
-v1.6.2 [latest release] [#stable]
+v2.0.1-5 [latest release] [#beta]
```
-### Upcoming:
-
-These are some modules which I have thought of adding:
-- Some more of Enumeraton & Information Disclosure modules.
-- Lots more of OSINT & Stuff (let that be a suspense).
-- More of Auxillary Modules.
-- Some Exploits are too being worked on.
-
-### Ongoing:
-
-- Working on a full-featured Web UI implementation on Flask and MongoDB and Node.js.
-- Working on a new framework, a real framework. `To be released with v2`
-- Working on a campaign feature + addition of arguments.
-- Normal Bug Fixing Stuffs. `As per the issues being raised`
-- Some other perks:
- - Working on a way for contributing new modules easily.
- - A complete new method of multi-threaded fuzzing of parameters.
- - Keeping better of new console stuff.
-
### Disclaimer:
-TIDoS is provided as a offensive web application audit framework. It has built-in modules which can reveal potential misconfigurations and vulnerabilties in web applications which could possibly be exploited maliciously.
-
-__THEREFORE, THE AUTHOR AND NEITHER THE CONTRIBUTORS ARE NOT EXCLUSIVELY RESPONSIBLE FOR ANY MISUSE OR DAMAGE DUE TO THIS TOOLKIT.__
-
-### Final Words:
-
-This project is a very fresh and new project which just simply springed off my mind, and is presently under active development so you may want to put it on a watch, since it is updated frequently.
-
-TIDoS is an in progress work far from perfection and I admit that there may be bugs out there which may cause many modules not to work properly and just bug out. However, being the only single author and maintainer behind this framework, it is my humble request to all users of this framework to hand me the list of modules via raising a [new issue](https://github.com/theInfectedDrake/TIDoS-Framework/issues/new) which simply do not work and bug out, and I would be more than happy to fix them as we jointly make our journey to realising TIDoS as the greatest web penetration testing framework ever built.
-
-> __TEMPORARY UPDATE__:
->
-> Please avoid doing any pull requests temporarily as work for v2 of this framework is already underway and in active development.
+TIDoS is provided as an offensive web application audit framework. It has built-in modules which can reveal potential misconfigurations and vulnerabilties in web applications which could possibly be exploited maliciously.
-Got more suggestions or new ideas? Raise up an [issue](https://github.com/theInfectedDrake/TIDoS-Framework/issues/new) or hit me up via DM on [twitter](https://twitter.com/infecteddrake).
+__THEREFORE, NEITHER THE AUTHOR NOR THE CONTRIBUTORS ARE RESPONSIBLE FOR ANY MISUSE OR DAMAGE DUE TO THIS TOOLKIT.__
-> Copyright © [__The Infected Drake__](https://twitter.com/infecteddrake)
diff --git a/core/.directory b/core/.directory
new file mode 100644
index 00000000..acdd027f
--- /dev/null
+++ b/core/.directory
@@ -0,0 +1,2 @@
+[Desktop Entry]
+Icon=/home/vainlystrain/Pictures/vaileava11.png
diff --git a/core/Auxillaries/__init__.py b/core/Auxillaries/__init__.py
deleted file mode 100644
index 73a0d197..00000000
--- a/core/Auxillaries/__init__.py
+++ /dev/null
@@ -1,11 +0,0 @@
-#!/usr/bin/env python2
-# coding: utf-8
-
-#-:-:-:-:-:-:-:-:-:-:-:-:#
-# TIDoS Framework #
-#-:-:-:-:-:-:-:-:-:-:-:-:#
-
-#This module requires TIDoS Framework
-#https://github.com/theInfectedDrake/TIDoS-Framework
-
-pass
diff --git a/core/Auxillaries/auxil.py b/core/Auxillaries/auxil.py
deleted file mode 100644
index c03a906d..00000000
--- a/core/Auxillaries/auxil.py
+++ /dev/null
@@ -1,76 +0,0 @@
-#!/usr/bin/env python2
-# coding: utf-8
-
-#-:-:-:-:-:-:-:-:-:-:-:-:#
-# TIDoS Framework #
-#-:-:-:-:-:-:-:-:-:-:-:-:#
-
-#This module requires TIDoS Framework
-#https://github.com/theInfectedDrake/TIDoS-Framework
-
-from __future__ import print_function
-import sys
-import os
-import time
-import subprocess
-import random
-from random import randint
-from subprocess import call
-sys.path.append('modules/0x05-Auxillaries+PF6/')
-from auxilban import *
-from encodeall import *
-from honeypot import *
-from hashes import *
-from imgext import *
-
-def auxil(web):
-
- auxilban()
- print('')
- time.sleep(0.3)
- v = raw_input(GR+' [#] \033[1;4mTID\033[0m'+GR+' :> ' + color.END)
- print('')
- if v == '1':
- print(' [!] Type Selected : Generate Hashes')
- hashes()
- print('\n\n')
- raw_input(O+' [+] Press '+GR+'Enter'+O+' to Continue...')
- auxilban()
- auxil(web)
-
- elif v == '2':
- print(' [!] Type Selected : Encode Strings')
- encodeall()
- print('\n\n')
- raw_input(O+' [+] Press '+GR+'Enter'+O+' to Continue...')
- auxilban()
- auxil(web)
-
- elif v == '3':
- print(' [!] Type Selected : Extract Metadata')
- imgext()
- print('\n\n')
- raw_input(O+' [+] Press '+GR+'Enter'+O+' to Continue...')
- auxilban()
- auxil(web)
-
- elif v == '4':
- print(' [!] Type Selected : Honeypot Detector')
- honeypot()
- print('\n\n')
- raw_input(O+' [+] Press '+GR+'Enter'+O+' to Continue...')
- auxilban()
- auxil(web)
-
- elif v == '99':
- print(GR+' [*] Going back!')
- time.sleep(0.7)
- os.system('clear')
-
- else:
- dope = ['You high dude?','Shit! Enter a valid option','Whoops! Thats not an option','Sorry! You just typed shit']
- print(dope[randint(0,3)])
- time.sleep(0.7)
- os.system('clear')
- auxilban()
- auxil(web)
diff --git a/core/Auxillaries/auxilban.py b/core/Auxillaries/auxilban.py
deleted file mode 100644
index d0f69535..00000000
--- a/core/Auxillaries/auxilban.py
+++ /dev/null
@@ -1,34 +0,0 @@
-#!/usr/bin/env python2
-# -*- coding: utf-8 -*-
-
-#-:-:-:-:-:-:-:-:-:-:-:-:#
-# TIDoS Framework #
-#-:-:-:-:-:-:-:-:-:-:-:-:#
-
-#This module requires TIDoS Framework
-#https://github.com/theInfectedDrake/TIDoS-Framework
-
-from __future__ import print_function
-import time
-import os
-from colors import *
-from core.Core.arts import auxilban_art
-
-def auxilban():
-
- os.system('clear')
- print(" [!] Module Selected : Auxillary Modules\n")
- time.sleep(0.4)
- print(auxilban_art)
- time.sleep(0.3)
- print('')
- print(B+' [1]'+C+' Generate Hashes from Strings'+W+'(4 Types) ')
- time.sleep(0.1)
- print(B+' [2]'+C+' Encode Payload or Strings'+W+' (7 Types)')
- time.sleep(0.1)
- print(B+' [3]'+C+' Extract Metadata from Images'+W+' (EXIF Data)')
- time.sleep(0.1)
- print(B+' [4]'+C+' HoneyPot Probability'+W+' (ShodanLabs HoneyScore)\n')
- time.sleep(0.1)
- print(B+' [99]'+C+' Back\n')
-
diff --git a/core/Auxillaries/colors.py b/core/Auxillaries/colors.py
deleted file mode 100644
index 96ed4433..00000000
--- a/core/Auxillaries/colors.py
+++ /dev/null
@@ -1,37 +0,0 @@
-#!/usr/bin/env python2
-
-#-:-:-:-:-:-:-:-:-:-:-:-:#
-# TIDoS Framework #
-#-:-:-:-:-:-:-:-:-:-:-:-:#
-
-#This module requires TIDoS Framework
-#https://github.com/theInfectedDrake/TIDoS-Framework
-
-###############################
-class color:
- PURPLE = '\033[95m'
- CYAN = '\033[96m'
- DARKCYAN = '\033[36m'
- BLUE = '\033[94m'
- GREEN = '\033[92m'
- YELLOW = '\033[93m'
- RED = '\033[91m'
- BOLD = '\033[1m'
- UNDERLINE = '\033[4m'
- END = '\033[0m'
- HEADER = '\033[95m'
- OKBLUE = '\033[94m'
- OKGREEN = '\033[92m'
- WARNING = '\033[93m'
- FAIL = '\033[91m'
-
-W = '\033[1;0m' # white (normal)
-R = '\033[1;31m' # red
-G = '\033[1;32m' # green
-O = '\033[1;33m' # orange
-B = '\033[1;34m' # blue
-P = '\033[1;35m' # purple
-C = '\033[1;36m' # cyan
-GR = '\033[1;37m' # gray
-T = '\033[1;93m' # tan
-###############################
diff --git a/core/Auxillaries/module_index.py b/core/Auxillaries/module_index.py
deleted file mode 100644
index e69de29b..00000000
diff --git a/core/Core/__init__.py b/core/Core/__init__.py
index 73a0d197..8d1c8b69 100644
--- a/core/Core/__init__.py
+++ b/core/Core/__init__.py
@@ -1,11 +1 @@
-#!/usr/bin/env python2
-# coding: utf-8
-
-#-:-:-:-:-:-:-:-:-:-:-:-:#
-# TIDoS Framework #
-#-:-:-:-:-:-:-:-:-:-:-:-:#
-
-#This module requires TIDoS Framework
-#https://github.com/theInfectedDrake/TIDoS-Framework
-
-pass
+
diff --git a/core/Core/__pycache__/__init__.cpython-37.pyc b/core/Core/__pycache__/__init__.cpython-37.pyc
new file mode 100644
index 00000000..646a3499
Binary files /dev/null and b/core/Core/__pycache__/__init__.cpython-37.pyc differ
diff --git a/core/Core/__pycache__/colors.cpython-37.pyc b/core/Core/__pycache__/colors.cpython-37.pyc
new file mode 100644
index 00000000..f4f04f36
Binary files /dev/null and b/core/Core/__pycache__/colors.cpython-37.pyc differ
diff --git a/core/Core/agree.py b/core/Core/agree.py
deleted file mode 100644
index 8492efff..00000000
--- a/core/Core/agree.py
+++ /dev/null
@@ -1,40 +0,0 @@
-#!/usr/bin/env python2
-# coding: utf-8
-
-#-:-:-:-:-:-:-:-:-:-:-:-:#
-# TIDoS Framework #
-#-:-:-:-:-:-:-:-:-:-:-:-:#
-
-#This module requires TIDoS Framework
-#https://github.com/theInfectedDrake/TIDoS-Framework
-
-from __future__ import print_function
-import os
-import sys
-import platform
-import time
-from agreement import *
-from colors import *
-
-def agree():
-
- os.system('clear')
- if str(platform.system()) != "Linux":
- sys.exit(R+" [!] " + color.UNDERLINE + "\033[91m" + "You are not using a Linux Based OS! Linux is a must-have for this script!" + color.END)
- if not os.geteuid() == 0:
- sys.exit(" [!] " + color.UNDERLINE + "\033[91m" + "Must be run as root. :) " + color.END)
- if 'no' in open('agree').read():
- agreement()
-
- a1 = raw_input(O+' [0x00] '+G+'Do you agree to these terms and conditions? :> '+C)
- if a1 == "yes" or a1 == 'y' or a1 == 'Y' or a1 == 'Yes' or a1 == 'yo' or a1 == 'YES' or a1 == 'yep' or a1 == 'Yep' or a1 == 'YEP':
- print(G+' [0x01] '+O+'Thats awesome! Move on...')
- time.sleep(3)
- FILE = open("agree","w")
- FILE.write('yes')
- FILE.close()
-
- else:
- print(O+' [0x0F] '+R+'You have to agree!')
- time.sleep(1)
- sys.exit(0)
diff --git a/core/Core/agreement.py b/core/Core/agreement.py
deleted file mode 100644
index 703e69d5..00000000
--- a/core/Core/agreement.py
+++ /dev/null
@@ -1,30 +0,0 @@
-#!/usr/bin/env python2
-# -*- coding: utf-8 -*-
-
-#-:-:-:-:-:-:-:-:-:-:-:-:#
-# TIDoS Framework #
-#-:-:-:-:-:-:-:-:-:-:-:-:#
-
-#This module requires TIDoS Framework
-#https://github.com/theInfectedDrake/TIDoS-Framework
-
-from __future__ import print_function
-def agreement():
-
- print("""
-\033[1;95m
- ==================================
- T I D O S F R A M E W O R K
- ==================================
-\033[1;37m
- TIDoS is an open-source toolkit developed as a comprehensive web-application penetration testing framework.
-
- TIDoS was developed by \033[1;36m@_tID\033[1;37m aka \033[1;36mThe Infected Drake.\033[1;37m
-
- This is for you to note that TIDoS Framework has capable modules in various phases which can unveil potential dangerous flaws in various web-applications, which can further be exploited maliciously. Therefore the author as well as the contrbutors assume no liability for misuse of this toolkit. Usage of TIDoS Framework for testing or exploiting websites without prior mutual consistency can be considered as an illegal activity. It is the end user's responsibility to obey all applicable local, state and federal laws.
-
- You can edit these scripts within this tool as per your own needs, provided you use it only for yourself. If you're using some source-code from here, give the developer some credits. :) \033[1;91m
-
- P.S. - If you find any bugs with this tool, report it.
-
-""")
diff --git a/core/Core/arts.py b/core/Core/arts.py
deleted file mode 100644
index 2886a406..00000000
--- a/core/Core/arts.py
+++ /dev/null
@@ -1,153 +0,0 @@
-#!/usr/bin/env python2
-# -*- coding: utf-8 -*-
-
-#-:-:-:-:-:-:-:-:-:-:-:-:#
-# TIDoS Framework #
-#-:-:-:-:-:-:-:-:-:-:-:-:#
-
-#Author : @_tID
-#This module requires TIDoS Framework
-#https://github.com/theInfectedDrake/TIDoS-Framework
-
-from colors import *
-
-auxilban_art = C+'''
-\033[1;37m
- ' . + . . ' + ' . .
- . ' + . * .
- + ' . +
- . * . . . * . *
- . \033[1;33m _\033[1;37m . . . .
- . . \033[1;33m _ / | \033[1;37m . . * \033[1;33m _ \033[1;37m . . +
- \033[1;33m| \_| | \033[1;37m + . \033[1;33m| | __
- \033[1;33m _ | |\033[1;37m . _ \033[1;33m| |/ |
- + \033[1;33m| \ | \033[1;36m _/\_ \033[1;33m| | / | \ \033[1;37m + /
- \033[1;33m| | \ \033[1;36m+/_\/_\+ \033[1;33m| | / | \ \033[1;37m . |\033[1;34m
- ____\033[1;33m/____\--...\___ \033[1;36m\_||_/\033[1;34m ___...\033[1;33m|__\_\033[1;34m..\033[1;33m|____\____/\033[1;34m_______/_\033[1;34m
- . . \033[1;36m|_|__|_| \033[1;34m . . .
- . . . \033[1;36m_/ /__\ \_\033[1;34m . . . .
- . . . . . .
- . ' ' ' '
- ' \033[1;33m-=[\033[1;31m A U X I L L A R I E S \033[1;33m]=-\033[1;34m . ' .
- . ' . . ' .
-'''
-
-scanenumban_art = """
- \033[1;33m,-. . \033[1;37m + . +
-\033[1;37m * \033[1;33m / \ `. \033[1;31m __..-,O \033[1;37m + * .
-\033[1;37m + \033[1;33m: \ \033[1;31m--''_..-'.'
-\033[1;37m \033[1;33m| . \033[1;31m.-' `. '.\033[1;37m + . . + +
-\033[1;37m . \033[1;33m: . \033[1;31m.`.' \033[1;37m
-\033[1;37m \033[1;33m \ `. \033[1;31m/ .. \033[1;37m . + + .
-\033[1;37m + \033[1;33m \ `. \033[1;31m' \033[1;33m. \033[1;37m * .
-\033[1;37m \033[1;33m `, `. \ \033[1;37m + +
-\033[1;37m . \033[1;36m,|,\033[1;33m`. `-.\ \033[1;37m * .
-\033[1;37m + \033[1;36m'.|| \033[1;33m ``-...__..-` \033[1;37m '
-\033[1;37m + \033[1;36m| | \033[1;37m . * +
-\033[1;37m * \033[1;36m|__| \033[1;37m + * .
-\033[1;37m . \033[1;36m/||\ \033[1;37m .
-\033[1;37m . \033[1;36m//||\＼ \033[1;37m + \033[1;33m-=[ \033[1;31mP R O B E & E N U M E R A T E \033[1;33m]=-\033[1;37m
-\033[1;37m + \033[1;36m// || \＼ \033[1;37m +
-\033[1;37m \033[1;36m __//__||__\＼_ \033[1;37m . . * . +
-\033[1;33m ____________\033[1;36m'--------------'\033[1;33m____________________________________________ """
-
-exploitsban_art = '''
- \033[1;33m|\ \033[1;33m/)
-\033[1;37m /\_\033[1;33m\＼\033[1;37m__ \033[1;33m(_//
-\033[1;37m| `>＼-` \033[1;36m_._ \033[1;33m//`)
-\033[1;37m \ /` \033[1;33m\＼\033[1;36m _.-\033[1;34m:::\033[1;36m`-._ \033[1;33m//
-\033[1;37m ` \033[1;33m\|\033[1;36m` \033[1;34m::: \033[1;36m `|\033[1;33m// \033[1;37m[0x00] \033[1;31mE X P L O I T S
-\033[1;36m | \033[1;34m ::: \033[1;36m|\033[1;33m/
-\033[1;36m |\033[1;34m.....:::.....\033[1;36m| \033[1;31mC A S T L E \033[1;37m[0x00]
-\033[1;36m |\033[1;34m:::::::::::::\033[1;36m|
-\033[1;36m | \033[1;34m::: \033[1;36m|
-\033[1;36m \ \033[1;34m ::: \033[1;36m/
-\033[1;36m \ \033[1;34m ::: \033[1;36m / \033[1;37m" [0x00] \033[1;33mCode the Exploits
-\033[1;36m `-. \033[1;34m:::\033[1;36m .-' \033[1;33mand
- \033[1;33m//\033[1;36m`:::`\033[1;33m\＼ \033[1;33mExploit the Codes \033[1;37m[0x00] "
- \033[1;33m// \033[1;36m' \033[1;33m\＼
- |/ \033[1;33m\| '''
-
-footprintban_art = """
-\033[1;36m
-
-\033[1;37m . .
- * . . . . * .
- . . . . . \033[1;35m ###
- \033[1;35mo \033[1;33m-=[ \033[1;31mR E C O N N A I S S A N C E \033[1;33m]=- \033[1;35m > ######-\033[1;37m --0
- + . . . \033[1;35m###
- \033[1;35m0\033[37m . . .
- . . + , , ,
- . \033[1;35m\ \033[37m . . + .
- . \033[1;35m\ \033[36m . . \033[1;36m###\033[1;37m .
- . \033[1;35mo \033[36m . \033[1;33m> \033[1;36m###########- --0\033[1;37m . +
- . \033[36m\ \033[1;36m########\033[1;37m . .
- \033[34m#\##\##. \033[1;33m> \033[1;36m###########- --0\033[1;37m . .
- + \033[34m# #O##\### \033[1;36m###\033[1;37m . + .
- . \033[34m#*# #\##\###\033[37m . + .
- . \033[34m##*# #\##\##\033[37m + .
- . \033[34m##*# #o##\# \033[37m . * , .
- . \033[34m**# #\# \033[37m . . .
- + \ . \033[33m/\^ \033[37m \033[1;33m.". \033[1;33m/
-\033[1;33m____^/\___^--____/\____\033[1;31mO\033[1;33m_____________/ \/\___________/\/ \______________
- /\^ ^ ^ ^ ^^ ^ '\ ^ ^ ---
- -- - -- - - --- __ ^
- -- __ ___-- ^ ^ -- __"""
-
-vulnban_art = R+"""
- .....
- .:noONNNNNNNOon:.
- .:NNNNNNNmddddNNNNNNN:.
- :NNNNmy+:. + .:+ymNNNN:
- NNNNy:` + `:yNNNN
- NNNNy. -!NNNN
- NNNN/ + \NNNN
- NNNm- .:#####:. -mNNN \033[1;37m[0x00] \033[1;33mV U L N E R A B I L I T Y \033[1;31m
- :NNN+ # + # +NNN:
- NNNm # + # mNNN \033[1;33mE N U M E R A T I O N\033[1;31m \033[1;37m[0x00]
- NNNh+++ ++#+++++++++++#++ +++hNNN
- NNNm # + # mNNN
- :NNN+ # + # +NNN:
- NNNm- *:#####:* -mNNN
- NNNN\ + /NNNN
- NNNNy. -yNNNN
- NNNNy:` + `:yNNNN"
- :NNNNmy+:. + .:+ymNNNN:
- *:NNNNNNNmddddNNNNNNNN*
- *:!NNNNNNNNNNN!:*
- '''*'''
-
-"""
-
-bugsban_art = C+'''
-\033[1;34m
- +------------------------------------------------------+
- | \033[1;37mTIDoS Dialog \033[1;33m[-] [口] [×] \033[1;34m|
- | ---------------------------------------------------- |
-\033[1;36m | |
- | \033[1;33mTIDoS has detected that you want to hunt for bugs! \033[1;36m |
- | \033[1;33mDo you wish to continue? \033[1;36m |
- | |
- | \033[1;37m.----------. .----------. .----------. \033[1;36m|
- | \033[1;37m| Yes \033[1;37m| | No \033[1;37m| | Maybe \033[1;37m| \033[1;36m|
- | \033[1;37m'----------' '----------' '----------' \033[1;36m |
- |______________________________________________________|
-
-'''
-
-tidosrules_art = """
- _nnnn_
- dGGGGMMb
- @p~qp~~qMb {}TIDoS Rules!!!
- M(\033[37m@\033[96m)(\033[37m@\033[96m) M| {}_;
- @\033[33m,----.\033[96mJM| {}-'
- JS^\033[33m\__/ \033[96mqKL
- dZP qKRb
- dZP qKKb
- fZP SMMb
- HZM MMMM
- FqM MMMM
- \033[33m_| '. |\033[96mdS'qML'
- \033[33m| `. | `' \_\033[96mZq'
- \033[33m_) \.___.,| .'
- \033[33m\________)\033[96mMMMMM\033[33m| .'""".format(O, GR, GR)
diff --git a/core/Core/banner.py b/core/Core/banner.py
deleted file mode 100644
index cbe2a977..00000000
--- a/core/Core/banner.py
+++ /dev/null
@@ -1,145 +0,0 @@
-# coding: utf-8
-#!/usr/bin/env python
-
-#-:-:-:-:-:-:-:-:-:-:-:-:#
-# TIDoS Framework #
-#-:-:-:-:-:-:-:-:-:-:-:-:#
-
-#This module requires TIDoS Framework
-#https://github.com/theInfectedDrake/TIDoS-Framework
-
-from __future__ import print_function
-import os
-from random import randint
-from colors import *
-
-def banner():
-
- header = """
-\033[1;31m
- T H E
-
- ▄▀▀▀█▀▀▄ ▄▀▀█▀▄ ▄▀▀█▄▄ ▄▀▀▀▀▄ ▄▀▀▀▀▄ \033[1;37m
- █ █ ▐ █ █ █ █ ▄▀ █ █ █ ██ ▐
- ▐ █ ▐ █ ▐ ▐ █ █ █ █ ▀▄
- █ █ █ █ ▀▄ ▄▀ ▀▄ █ \033[1;31m
- ▄▀ ▄▀▀▀▀▀▄ ▄▀▄▄▄▄▀ ▀▀▀▀ █▀▀▀
- █ █ █ █ ▐ ▐
- ▐ ▐ ▐ ▐
- F R A M E W O R K
-
-"""
-
-
- oblique = C + """
-\033[1;36m
- T H E
-
- ███ ▄█ ████████▄ ▄██████▄ ▄████████
- ▀█████████▄ ███ ███ ▀███ ███ ███ ███ ███ \033[1;37m
- ▀███▀▀██ ███▌ ███ ███ ███ ███ ███ █▀
- ███ ▀ ███▌ ███ ███ ███ ███ ███
- ███ ███▌ ███ ███ ███ ███ ▀███████████
- ███ ███ ███ ███ ███ ███ ███ \033[1;36m
- ███ ███ ███ ▄███ ███ ███ ▄█ ███
- ▄████▀ █▀ ████████▀ ▀██████▀ ▄████████▀
-
- F R A M E W O R K
-"""
-
- modular = O + """
-\033[1;33m
- ___________________________
- |\_________________________/|＼
- || || ＼
- || \033[1;36mThe \033[1;33m|| |
- || \033[1;36m TIDoS \033[1;33m|| |
- || \033[1;36m Framework \033[1;33m|| |
- || \033[1;33m|| |
- || \033[1;34mWeb Application Audit \033[1;33m|| |
- || \033[1;34m Framework \033[1;33m|| |
- || \033[1;33m|| |
- || \033[1;37mFrom: CodeSploit \033[1;33m|| /
- ||_________________________|| /
- |/_________________________\|/
- __\_________________/__/|
- |_______________________|/
- ________________________
- /\033[0m\033[37moooo oooo oooo oooo \033[1;33m/|
- /\033[0m\033[37mooooooooooooooooooooooo\033[1;33m/ /
- /\033[0m\033[37mooooooooooooooooooooooo\033[1;33m/ /
- /C=_____________________/_/
-"""
-
- fb = C + """
-\033[1;36m
- ,------,-------------------------------------,------.
- | | | |
- | | \033[1;37mThe TIDoS Framework | |
- | | \033[1;34m< CodeSploit > \033[1;36m | |
- | | | |
- | |-------------------------------------| |
- | ||...................................|| |
- | || || |
- | || \033[1;33mHack Facebook? (y/n) \033[1;36m|| |
- | || \033[1;34m---------------------- \033[1;36m|| |
- | ||_______ \033[1;33m$ > (Input) \033[1;36m_______|| |
- | || \033[1;32m$$$$$ \033[1;36m| | \033[1;32m$$$$$\033[1;36m || |
- | ||-------'-------------------'-------|| |
- `------'|___________________________________|`------'
-
-"""
- codesploit = GR + """
-
- MMMMMNMNMMMM$%&.
- .DMMM lM$%.
- .MMN \M$%,.
- MN \M%$..
- .M. \033[1;36mXXXXXXXXXXXXX \033[1;37m\M%$M
- .M \033[1;36m XXXXXXXXXXXXX \033[1;37m\l$%M.
- M \033[1;36m XXX \033[1;37m\M$%M:.
- M \033[1;36m XXX \033[1;37m\$MMM:
- M \033[1;36m XXX \033[1;34mXXXXXXX \033[1;37m:$%MM:
- :M \033[1;36m XXX \033[1;34mXX \033[1;37m :$%MMM:
- M \033[1;36m XXX \033[1;34mXXXXXXX \033[1;37m M%$MM:
- :M: \033[1;36m XXX \033[1;34m XX \033[1;37m M$%M:'
- NM \033[1;36m XXXXXX\033[1;34mXXXXXXX\033[1;36mm \033[1;37m .M$%MM:
- IMM. \033[1;36m XXXXXXXXXXXXX\033[1;36mm \033[1;37m.MD%4M'
- :MM . .MM$%M'
- $MM .MMM$%"'
- MMMM. MMMMMM"
- MMMMMMMMMMMMMMMMMM:*"
-\033[1;33m
- ╔═════════════════════════╗
- █ \033[1;31mC O D E S P L O I T\033[1;33m █
- ╚═════════════════════════╝
-"""
- swan = """
-\033[1;33m
- ...,ooooooooo......
- .o8888888888888888888888888o.
- .o888888888888888888888888888888888o.
- o8888888888A88"V888888888888888888888888o
- o88888887"8" V V888 88888888888888888888o
- o88888888 VV0 888888888888888888888
- o888888888 !/ 88888888888888888888o
- .88888888888 88888V" "V8888888888.
- o88888888888v \033[1;37mT H E \033[1;33m 8888" v8 8888888888o
- 88888888888v 8888v v88 88888888888
- 888888888888 \033[1;37mT I D O S\033[1;33m 88888v "8888888888888
- 88888888888V ""M888 "888P8888888
- 88888888888v \033[1;37mF R A M E W O R K \033[1;33m 88P8888888
-\033[1;36m ______\033[1;33m8888888888888v.........................VF8888888\033[1;36m_______
- :::::::::::::::::::' ::::::::::::::::
- ::::::::::::::::: .::::::: .:::::::::::::::::\033[1;34m
- ::::::::::::::: \033[1;36mFrom:\033[1;34m ::::::: .:::::::::::::::::::
- ::::::::::::::: \033[1;37mCodeSploit\033[1;34m :::::: ::: ::::::::::::::::
- :::::::::::::::. ::::::. :: .::::::::::::::::\033[1;36m
- :::::::::::::::: :::::::. .:::::::::::::::::
- :::::::::::::::. . ::::::::::::::::::::::::::::
-"""
- headers = [header, oblique, modular, codesploit, fb, swan]
- os.system('clear')
- print(headers[randint(0,5)])
-
-
diff --git a/core/Core/bannerbelow.py b/core/Core/bannerbelow.py
deleted file mode 100644
index abd2bab9..00000000
--- a/core/Core/bannerbelow.py
+++ /dev/null
@@ -1,33 +0,0 @@
-#!/usr/bin/env python2
-# -*- coding: utf-8 -*-
-
-#-:-:-:-:-:-:-:-:-:-:-:-:#
-# TIDoS Framework #
-#-:-:-:-:-:-:-:-:-:-:-:-:#
-
-#This module requires TIDoS Framework
-#https://github.com/theInfectedDrake/TIDoS-Framework
-
-from __future__ import print_function
-import time
-from time import sleep
-from colors import *
-
-def bannerbelow():
-
- print(B+'[---] '+C+'The TIDoS Framework \033[36m| \033[1;37mVersion v1.6 \033[1;34m[---]')
- sleep(0.2)
- print(B+'[---] [---]')
- print(B+'[---] \033[1;31m~ Author \033[1;31m: \033[1;33m@_tID ~ \033[1;34m[---]')
- sleep(0.2)
- print(B+'[---] '+O+'~ github.com / \033[1;32mtheInfectedDrake ~ \033[1;34m[---]')
- sleep(0.2)
- print(B+'[---] [---]')
- sleep(0.2)
- print(B+'[---] \033[1;35m5 Phases | \033[1;31m14 Sub-Phases | \033[1;37m108 Modules \033[1;34m[---]')
- sleep(0.2)
- print('')
- print(B+' Welcome to '+C+'The TIDoS Framework (TTF)')
- sleep(0.2)
- print(GR+' The TIDoS Framework is a project by '+R+'Team CodeSploit\n')
-
diff --git a/core/Core/colors.py b/core/Core/colors.py
index 96ed4433..1a278a6f 100644
--- a/core/Core/colors.py
+++ b/core/Core/colors.py
@@ -1,37 +1,64 @@
-#!/usr/bin/env python2
+#!/usr/bin/env python3
#-:-:-:-:-:-:-:-:-:-:-:-:#
# TIDoS Framework #
#-:-:-:-:-:-:-:-:-:-:-:-:#
#This module requires TIDoS Framework
-#https://github.com/theInfectedDrake/TIDoS-Framework
+#https://github.com/0xInfection/TIDoS-Framework
###############################
class color:
- PURPLE = '\033[95m'
- CYAN = '\033[96m'
- DARKCYAN = '\033[36m'
- BLUE = '\033[94m'
- GREEN = '\033[92m'
- YELLOW = '\033[93m'
- RED = '\033[91m'
- BOLD = '\033[1m'
- UNDERLINE = '\033[4m'
- END = '\033[0m'
- HEADER = '\033[95m'
- OKBLUE = '\033[94m'
- OKGREEN = '\033[92m'
- WARNING = '\033[93m'
- FAIL = '\033[91m'
+ PURPLE = '\033[95m'
+ CYAN = '\033[96m'
+ DARKCYAN = '\033[36m'
+ BLUE = '\033[94m'
+ GREEN = '\033[92m'
+ YELLOW = '\033[93m'
+ RED = '\033[91m'
+ BOLD = '\033[1m'
+ UNDERLINE = '\033[4m'
+ CURSIVE = '\033[3m'
+ END = '\033[0m'
+ HEADER = '\033[95m'
+ OKBLUE = '\033[94m'
+ OKGREEN = '\033[92m'
+ WARNING = '\033[93m'
+ FAIL = '\033[91m'
+ #TR1 = '\033[0m\033[48;2;58;49;58m\033[38;2;225;214;225m\uE0B0'
+ #TR2 = '\033[0m\033[38;2;225;214;225m\uE0B0'
+ #TR3 = '\033[0m\033[48;2;225;214;225m\033[38;2;58;49;58m\uE0B0'
+ #TR4 = '\033[0m\033[38;2;58;49;58m\uE0B0'
+ #TR5 = '\033[0m\033[48;2;225;214;225m\033[38;2;19;14;19m\uE0B0'
+ #TR6 = '\033[0m\033[48;2;58;49;58m\033[38;2;19;14;19m\uE0B0'
-W = '\033[1;0m' # white (normal)
-R = '\033[1;31m' # red
-G = '\033[1;32m' # green
-O = '\033[1;33m' # orange
-B = '\033[1;34m' # blue
-P = '\033[1;35m' # purple
-C = '\033[1;36m' # cyan
-GR = '\033[1;37m' # gray
-T = '\033[1;93m' # tan
+ TR1 = '\033[0m\033[48;2;58;49;58m\033[38;2;225;214;225m|'
+ TR2 = '\033[0m\033[38;2;225;214;225m|'
+ TR3 = '\033[0m\033[48;2;225;214;225m\033[38;2;58;49;58m|'
+ TR4 = '\033[0m\033[38;2;58;49;58m|'
+ TR5 = '\033[0m\033[48;2;225;214;225m\033[38;2;19;14;19m|'
+ TR6 = '\033[0m\033[48;2;58;49;58m\033[38;2;19;14;19m|'
+ VBG = '\033[48;2;19;14;19m'
+ TITLE = '\033[38;2;85;72;85m'
+ BAR = '\033[38;2;225;214;225m'
+ #TR1 = '\033[48;2;58;49;58m\033[38;2;225;214;225m\u25B6'
+
+W = '\033[0m\033[1;0m' # white (normal)
+#R = '\033[1;31m' # red
+R = '\033[0m\033[38;2;58;49;58m\033[1m'
+#G = '\033[0m\033[1m' # green
+#G = '\033[0m\033[48;2;255;255;255m\033[38;2;58;49;58m\033[1m'
+G = '\033[0m\033[48;2;225;214;225m\033[38;2;58;49;58m\033[1m'
+#G = '\033[0m\033[48;2;85;72;85m\033[38;2;225;214;225m'
+#O = '\033[0m\033[1m' # orange
+O = '\033[0m\033[48;2;58;49;58m'
+B = '\033[0m\033[1m' # blue
+#P = '\033[0m\033[1m' # purple
+P = '\033[0m\033[38;2;58;49;58m\033[3m'
+C = '\033[0m\033[1m' # cyan
+GR = '\033[0m\033[1m' # gray
+T = '\033[0m\033[1m' # tan
+RB = '\033[48;2;58;49;58m'
+RC = '\033[0m\033[38;2;58;49;58m\033[3m'
+RD = '\033[0m\033[38;2;58;49;58m'
###############################
diff --git a/core/Core/dispmenu.py b/core/Core/dispmenu.py
deleted file mode 100644
index d40ed2b0..00000000
--- a/core/Core/dispmenu.py
+++ /dev/null
@@ -1,50 +0,0 @@
-#!/usr/bin/env python2
-# coding: utf-8
-
-#-:-:-:-:-:-:-:-:-:-:-:-:#
-# TIDoS Framework #
-#-:-:-:-:-:-:-:-:-:-:-:-:#
-
-#This module requires TIDoS Framework
-#https://github.com/theInfectedDrake/TIDoS-Framework
-
-from __future__ import print_function
-import time
-from colors import *
-
-def dispmenu():
-
- print('''
-
-\033[1;37m . + \033[1;34m ______ \033[1;37m . .
- +. \033[1;34m / ==== \ \033[1;37m . + . .
- . . \033[1;36m ,-~--------~-. \033[1;37m * +
- \033[1;36m,^\033[1;33m ___ \033[1;36m^.\033[1;37m + * . . .
- * * \033[1;36m / \033[1;33m.^ ^. \033[1;36m\ \033[1;37m . \033[1;32m _ | _
- \033[1;36m| \033[1;33m| o ! \033[1;36m|\033[1;37m . \033[1;32m __ \ /--.
- . \033[1;36m|\033[1;34m_ \033[1;33m'.___.' \033[1;34m_\033[1;36m|\033[1;37m \033[1;32mI__/_\ / )}\033[1;36m======> \033[1;37m +
- \033[1;36m| \033[1;34m"'----------------"\033[1;36m|\033[1;37m + \033[1;32m _[ _(\033[1;33m0\033[1;32m): ))\033[1;36m========>
- + . \033[1;36m! !\033[1;37m . \033[1;32mI__\ / \. ]}\033[1;36m======> \033[1;37m .
- . \033[1;36m \ \033[1;37mTIDoS Prober \033[1;36m/ \033[1;37m \033[1;32m ~^-.--'
- \033[1;36m^. .^ \033[1;37m . \033[1;32m | \033[1;37m +. *
- . \033[1;36m "-..______.,-" \033[1;37m. . *
- + . . + * .
- \033[1;33m-=[ \033[1;31mL E T S S T A R T\033[1;33m ]=-\033[1;37m
- + . ' . + +
- * . + * . * .
-''')
- print(O+'\n Choose from the options below :\n')
- time.sleep(0.2)
- print(B+' [1] \033[1;36mReconnaissance & OSINT'+W+' (50 modules)')
- time.sleep(0.1)
- print(B+' [2] \033[1;36mScanning & Enumeration'+W+' (16 modules)')
- time.sleep(0.1)
- print(B+' [3] \033[1;36mVulnerability Analysis'+W+' (37 modules)')
- time.sleep(0.1)
- print(B+' [4] \033[1;36mExploitation (beta)'+W+' (1 modules)')
- time.sleep(0.1)
- print(B+' [5] \033[1;36mAuxillary Modules'+W+' (4 modules)\n')
- time.sleep(0.1)
- print(B+' [99] \033[1;36mSay "alvida"! (Exit TIDoS)\n')
- time.sleep(0.1)
-
diff --git a/core/Core/inputin.py b/core/Core/inputin.py
deleted file mode 100644
index 6f2d3956..00000000
--- a/core/Core/inputin.py
+++ /dev/null
@@ -1,58 +0,0 @@
-#!/usr/bin/env python2
-# coding: utf-8
-
-#-:-:-:-:-:-:-:-:-:-:-:-:#
-# TIDoS Framework #
-#-:-:-:-:-:-:-:-:-:-:-:-:#
-
-#This module requires TIDoS Framework
-#https://github.com/theInfectedDrake/TIDoS-Framework
-
-from __future__ import print_function
-import sys
-import socket
-import time
-import os
-from colors import *
-
-def inputin():
-
- try:
- web = raw_input(''+O+' [#] Target web address :> '+C)
- global web
- if not str(web).startswith('http'):
- mo = raw_input(GR+' [#] Does this website use SSL? (y/n) :> ')
- if mo == 'y' or mo == 'Y':
- web = 'https://'+web
- elif mo == 'n':
- web = 'http://'+web
- if 'http://' in web:
- po = web.split('//')[1]
- elif 'https://' in web:
- po = web.split('//')[1]
- if str(web).endswith('/'):
- web = po[:-1]
- po = po[:-1]
- print(GR+' [*] Checking server status...')
- time.sleep(0.6)
-
- try:
- ip = socket.gethostbyname(po)
- print(G+' [+] Site seems to be up...')
- time.sleep(0.5)
- print(G+' [+] IP Detected : '+O+ip)
- time.sleep(0.5)
- print('')
- os.system('cd tmp/logs/ && rm -rf '+po+'-logs && mkdir '+po+'-logs/')
- return web
-
- except socket.gaierror:
- print(R+' [-] Site seems to be down...')
- sys.exit(1)
-
- except KeyboardInterrupt:
- print(R+' [-] Exiting...')
- time.sleep(0.7)
- print(C+' [#] Alvida, my friend!')
- sys.exit(1)
-
diff --git a/core/Core/loadstyle.py b/core/Core/loadstyle.py
deleted file mode 100644
index bf8fdea4..00000000
--- a/core/Core/loadstyle.py
+++ /dev/null
@@ -1,35 +0,0 @@
-#!/usr/bin/env python2
-
-#-:-:-:-:-:-:-:-:-:-:-:-:#
-# TIDoS Framework #
-#-:-:-:-:-:-:-:-:-:-:-:-:#
-
-#This module requires TIDoS Framework
-#https://github.com/theInfectedDrake/TIDoS-Framework
-
-from __future__ import print_function
-import os
-import time
-from colors import *
-
-def loadstyle():
-
- os.system('clear')
- red_bold = R
- reset = W
- loading = "Loading the TIDoS Framework..."
- action = 0
- while action < 1:
- for i,char in enumerate(loading):
- if i == 0:
- print("%s%s%s%s" %(red_bold,char.upper(),reset,loading[1:]))
- elif i == 1:
- old_loading = loading[0].lower()
- print("%s%s%s%s%s" %(old_loading,red_bold,char.upper(),reset,loading[2:]))
- elif i == i:
- old_loading = loading[-0:i].lower()
- print("%s%s%s%s%s" %(old_loading,red_bold,char.upper(),reset,loading[i+1:]))
- time.sleep(0.1)
- os.system('clear')
- action += 1
-
diff --git a/core/Enumeration/Crawling/__init__.py b/core/Enumeration/Crawling/__init__.py
deleted file mode 100644
index 73a0d197..00000000
--- a/core/Enumeration/Crawling/__init__.py
+++ /dev/null
@@ -1,11 +0,0 @@
-#!/usr/bin/env python2
-# coding: utf-8
-
-#-:-:-:-:-:-:-:-:-:-:-:-:#
-# TIDoS Framework #
-#-:-:-:-:-:-:-:-:-:-:-:-:#
-
-#This module requires TIDoS Framework
-#https://github.com/theInfectedDrake/TIDoS-Framework
-
-pass
diff --git a/core/Enumeration/Crawling/crawlers.py b/core/Enumeration/Crawling/crawlers.py
deleted file mode 100644
index 6dbbe1bc..00000000
--- a/core/Enumeration/Crawling/crawlers.py
+++ /dev/null
@@ -1,84 +0,0 @@
-#!/usr/bin/env python2
-# -*- coding : utf-8
-
-#-:-:-:-:-:-:-:-:-:-:-:-:#
-# TIDoS Framework #
-#-:-:-:-:-:-:-:-:-:-:-:-:#
-
-#This script is a part of TIDoS Framework
-#https://github.com/theInfectedDrake/TIDoS-Framework
-
-from __future__ import print_function
-import sys
-import platform
-import os
-import time
-import random
-import subprocess
-from random import *
-sys.path.append('modules/0x02-Scanning+Enumeration/0x02-WebCrawling/')
-
-# Import stuff
-from crawler1 import *
-from crawler2 import *
-from crawler3 import *
-from crawlersban import *
-from colors import *
-
-def crawlers(web):
-
- time.sleep(0.3)
- print(' [!] Module Selected : Crawlers')
- time.sleep(0.4)
- crawlersban() # banner
- v = raw_input(O+' [#] TID :> ')
-
- if v.strip() == '1': # level 1 crawler
- print(B+' [!] Module Selected :'+C+' Crawler (Depth 1)')
- crawler1(web)
- time.sleep(1)
- crawlers(web)
-
- elif v.strip() == '2': # level 2 crawler
- print(B+' [!] Module Selected :'+C+' Crawler (Depth 2)')
- crawler2(web)
- time.sleep(1)
- crawlers(web)
-
- elif v.strip() == '3': # level 3 crawler
- print(B+' [!] Module Selected :'+C+' Crawler (Depth 3)')
- crawler3(web)
- time.sleep(1)
- crawlers(web)
-
- elif v.strip() == '99': # to go back
- print(GR+' [*] Going back...')
- time.sleep(0.5)
- os.system('clear')
-
- elif v.strip() == 'A':
- print(W+'\n [!] Module Automater Initialized...')
- sleep(0.5)
- print(B+' [*] Initializing Scan Type :'+C+' Crawler (Depth 1)')
- crawler1(web) # unleash level 1
-
- print(B+'\n [!] Scan Type Completed :'+C+' Crawler 1\n')
- sleep(0.5)
- print(B+' [!] Initializing Scan Type :'+C+' Crawler (Depth 2)')
- crawler2(web) # unleash level 2
-
- print(B+'\n [!] Scan Type Completed :'+C+' Crawler 2\n')
- sleep(0.5)
- print(B+' [!] Initializing Scan Type :'+C+' Crawler (Depth 3)')
- crawler3(web) # unleash level 3
-
- print(B+'\n [!] Scan Type Completed :'+C+' Crawler 3\n\n'+G+' [+] All modules successfully completed!')
- raw_input(GR+' [+] Press '+O+'Enter '+GR+'to continue...') # for the user to take a breadth :/
- crawlers(web) # get back to menu
-
- else:
- dope = ['You high dude?','Shit! Enter a valid option','Whoops! Thats not an option','Sorry! You just typed shit']
- print(' [-] '+dope[randint(0,3)]) # troll stuffs :p
- sleep(1)
- crawlers(web)
-
diff --git a/core/Enumeration/Crawling/crawlersban.py b/core/Enumeration/Crawling/crawlersban.py
deleted file mode 100644
index 3475a6bf..00000000
--- a/core/Enumeration/Crawling/crawlersban.py
+++ /dev/null
@@ -1,32 +0,0 @@
-#!/usr/bin/env python2
-# -*- coding : utf-8
-
-#-:-:-:-:-:-:-:-:-:-:-:-:#
-# TIDoS Framework #
-#-:-:-:-:-:-:-:-:-:-:-:-:#
-
-#This script is a part of TIDoS Framework
-#https://github.com/theInfectedDrake/TIDoS-Framework
-
-from __future__ import print_function
-import time
-from time import sleep
-from colors import *
-
-def crawlersban():
-
- print(O+'\n +-----------------+')
- print(O+' | '+G+' CRAWLER TYPES '+O+' |')
- print(O+' +-----------------+\n')
- print('')
- sleep(0.2)
- print(B+" [1] "+C+"Crawler \033[1;0m(Depth 1)")
- sleep(0.2)
- print(B+' [2] '+C+'Crawler \033[1;0m(Depth 2) ')
- sleep(0.2)
- print(B+' [3] '+C+'Crawler \033[1;0m(Depth 3) ')
- sleep(0.2)
- print(B+'\n [A] '+C+'Test both crawlers 1 by 1 ')
- sleep(0.2)
- print(B+'\n [99] '+C+'Back \n')
-
diff --git a/core/Enumeration/PortScans/__init__.py b/core/Enumeration/PortScans/__init__.py
deleted file mode 100644
index 73a0d197..00000000
--- a/core/Enumeration/PortScans/__init__.py
+++ /dev/null
@@ -1,11 +0,0 @@
-#!/usr/bin/env python2
-# coding: utf-8
-
-#-:-:-:-:-:-:-:-:-:-:-:-:#
-# TIDoS Framework #
-#-:-:-:-:-:-:-:-:-:-:-:-:#
-
-#This module requires TIDoS Framework
-#https://github.com/theInfectedDrake/TIDoS-Framework
-
-pass
diff --git a/core/Enumeration/PortScans/portscan.py b/core/Enumeration/PortScans/portscan.py
deleted file mode 100644
index 46f18fc5..00000000
--- a/core/Enumeration/PortScans/portscan.py
+++ /dev/null
@@ -1,155 +0,0 @@
-#!/usr/bin/env python2
-# -*- coding: utf-8 -*-
-
-#-:-:-:-:-:-:-:-:-:-:-:-:#
-# TIDoS Framework #
-#-:-:-:-:-:-:-:-:-:-:-:-:#
-
-#This script is a part of TIDoS Framework
-#https://github.com/theInfectedDrake/TIDoS-Framework
-
-from __future__ import print_function
-import sys
-import os
-import time
-import random
-from random import *
-from time import sleep
-sys.path.append('modules/0x02-Scanning+Enumeration/0x01-PortScanning/')
-
-from finscan import *
-from servicedetect import *
-#from nullscan import *
-#from tcpack import *
-from simpleport import *
-from tcpconnectscan import *
-from tcpstealthscan import *
-#from tcpwindows import *
-#from udpscan import *
-from xmasscan import *
-from colors import *
-from portscanban import *
-
-def portscan(web):
-
- time.sleep(0.3)
- print(W+' [!] Module Selected : Port Scanning')
- time.sleep(0.4)
- portscanban()
- v = raw_input (GR+' [#] \033[1;4mTID\033[0m'+GR+' :> ' + color.END)
-
- if v.strip() == '1':
- print(B+' [!] Module Selected :'+C+' Simple Port Scan')
- simpleport(web)
- time.sleep(1)
- portscan(web)
-
- elif v.strip() == '2':
- print(B+' [!] Module Selected :'+C+' TCP Connect Scan')
- tcpconnectscan(web)
- time.sleep(1)
- portscan(web)
-
-# elif v.strip() == '3':
-# print B+' [!] Module Selected :'+C+' TCP ACK Scan'
-# tcpack(web)
-# time.sleep(1)
-# portscan(web)
-
- elif v.strip() == '4':
- print(B+' [!] Module Selected :'+C+' TCP Stealth Scan')
- tcpstealthscan(web)
- time.sleep(1)
- portscan(web)
-
-# elif v.strip() == '5':
-# print B+' [!] Module Selected :'+C+' UDP Scan'
-# udpscan(web)
-# time.sleep(1)
-# portscan(web)
-
- elif v.strip() == '6':
- print(B+' [!] Module Selected :'+C+' XMAS Scan')
- xmasscan(web)
- time.sleep(1)
- portscan(web)
-
-# elif v.strip() == '7':
-# print B+' [!] Module Selected :'+C+' NULL Scan'
-# nullscan(web)
-# time.sleep(1)
-# portscan(web)
-
- elif v.strip() == '8':
- print(B+' [!] Module Selected :'+C+' FIN Scan')
- finscan(web)
- time.sleep(1)
- portscan(web)
-
- elif v.strip() == '9':
- print(B+' [!] Module Selected :'+C+' Service Detector')
- servicedetect(web)
- time.sleep(1)
- portscan(web)
-
-# elif v.strip() == '10':
-# print B+' [!] Module Selected :'+C+' TCP Windows'
-# tcpwindowsscan(web)
-# time.sleep(1)
-# portscan(web)
-
- elif v.strip() == 'A':
- print('\n [!] Module Automater Initialized...')
- sleep(0.5)
- print(B+' [*] Initializing Scan Type :'+C+' Simple Port Scan')
- simpleport(web)
- print(B+' [!] Scan Type Completed :'+C+' Simple Port Scan\n')
- sleep(0.5)
- print(B+' [!] Initializing Scan Type :'+C+' TCP Connect Scan')
- tcpconnectscan(web)
- print(B+' [!] Scan Type Completed :'+C+' TCP Connect\n')
- sleep(0.5)
-# print B+' [!] Initializing Scan Type :'+C+' TCP ACK Scan'
-# tcpack(web)
-# print B+' [!] Scan Type Completed :'+C+' TCP ACK Scan\n'
-# sleep(0.5)
- print(B+' [!] Initializing Scan Type :'+C+' TCP Stealth Scan')
- tcpstealthscan(web)
- print(B+' [!] Scan Type Completed :'+C+' TCP Stealth Scan\n')
- sleep(0.5)
-# print B+' [!] Initializing Scan Type :'+C+' UDP Scan'
-# udpscan(web)
-# print B+' [!] Scan Type Completed :'+C+' UDP Scan\n'
-# sleep(0.5)
- print(B+' [!] Initializing Scan Type :'+C+' XMAS Scan')
- xmasscan(web)
- print(B+' [!] Scan Type Completed :'+C+' XMAS Scan\n')
- sleep(0.5)
-# print B+' [!] Initializing Scan Type :'+C+' NULL Scan'
-# nullscan(web)
-# print B+' [!] Scan Type Completed :'+C+' NULL Scan\n'
-# sleep(0.5)
- print(B+' [!] Initializing Scan Type :'+C+' FIN Scan')
- finscan(web)
- print(B+' [!] Scan Type Completed :'+C+' FIN Scan\n')
- sleep(0.5)
- print(B+' [!] Initializing Scan Type :'+C+' Service Detection')
- servicedetect(web)
- print(B+' [!] Scan Type Completed :'+C+' Service Detection\n')
- sleep(0.5)
- print(B+' [!] All scantypes have been tested on target...')
- print(C+' [!] Going back to menu...')
- sleep(3)
- portscan(web)
-
- elif v.strip() == '99':
- print(GR+' [*] Going back...')
- time.sleep(0.5)
- os.system('clear')
-
- else:
- dope = ['You high dude?','Shit! Enter a valid option','Whoops! Thats not an option','Sorry! You just typed shit']
- print(dope[randint(0,3)])
- time.sleep(0.7)
- portscan(web)
-
diff --git a/core/Enumeration/PortScans/portscanban.py b/core/Enumeration/PortScans/portscanban.py
deleted file mode 100644
index 7998cdc8..00000000
--- a/core/Enumeration/PortScans/portscanban.py
+++ /dev/null
@@ -1,46 +0,0 @@
-#!/usr/bin/env python2
-# -*- coding : utf-8
-
-#-:-:-:-:-:-:-:-:-:-:-:-:#
-# TIDoS Framework #
-#-:-:-:-:-:-:-:-:-:-:-:-:#
-
-#This script is a part of TIDoS Framework
-#https://github.com/theInfectedDrake/TIDoS-Framework
-
-from __future__ import print_function
-import time
-import os
-from time import sleep
-from colors import *
-
-def portscanban():
-
- print(O+'\n +--------------+')
- print(O+' | '+W+'SCAN TYPES'+O+' |')
- print(O+' +--------------+\n')
- sleep(0.1)
- print(B+" [1]"+C+" A Simple Port Scan")
- sleep(0.1)
- print(B+" [2]"+C+" A TCP Connect Scan"+G+' (Highly Reliable)')
- sleep(0.1)
-# print B+" [3]"+C+" A TCP-ACK Scan"
-# sleep(0.1)
- print(B+" [4]"+C+" A TCP Stealth Scan"+G+' (Highly Reliable)')
- sleep(0.1)
-# print B+" [5]"+C+" A UDP Scan"
-# sleep(0.1)
- print(B+" [6]"+C+" A XMAS Flag Scan "+R+"(Reliable only in LANs)")
- sleep(0.1)
-# print B+" [7]"+C+" A NULL Scan"
-# sleep(0.1)
- print(B+" [8]"+C+" A FIN Flag Scan "+R+"(Reliable only in LANs)")
- sleep(0.1)
- print(B+" [9]"+C+" A Open Ports Services Detector")
- sleep(0.1)
-# print O+B+" [10]"+C+" A TCP Windows Scan"
-# sleep(0.1)
- print(B+" [A]"+C+" Automate all modules one by one\n")
- sleep(0.1)
- print(B+' [99]'+C+' Back\n')
-
diff --git a/core/Enumeration/__init__.py b/core/Enumeration/__init__.py
deleted file mode 100644
index 73a0d197..00000000
--- a/core/Enumeration/__init__.py
+++ /dev/null
@@ -1,11 +0,0 @@
-#!/usr/bin/env python2
-# coding: utf-8
-
-#-:-:-:-:-:-:-:-:-:-:-:-:#
-# TIDoS Framework #
-#-:-:-:-:-:-:-:-:-:-:-:-:#
-
-#This module requires TIDoS Framework
-#https://github.com/theInfectedDrake/TIDoS-Framework
-
-pass
diff --git a/core/Enumeration/colors.py b/core/Enumeration/colors.py
deleted file mode 100644
index 96ed4433..00000000
--- a/core/Enumeration/colors.py
+++ /dev/null
@@ -1,37 +0,0 @@
-#!/usr/bin/env python2
-
-#-:-:-:-:-:-:-:-:-:-:-:-:#
-# TIDoS Framework #
-#-:-:-:-:-:-:-:-:-:-:-:-:#
-
-#This module requires TIDoS Framework
-#https://github.com/theInfectedDrake/TIDoS-Framework
-
-###############################
-class color:
- PURPLE = '\033[95m'
- CYAN = '\033[96m'
- DARKCYAN = '\033[36m'
- BLUE = '\033[94m'
- GREEN = '\033[92m'
- YELLOW = '\033[93m'
- RED = '\033[91m'
- BOLD = '\033[1m'
- UNDERLINE = '\033[4m'
- END = '\033[0m'
- HEADER = '\033[95m'
- OKBLUE = '\033[94m'
- OKGREEN = '\033[92m'
- WARNING = '\033[93m'
- FAIL = '\033[91m'
-
-W = '\033[1;0m' # white (normal)
-R = '\033[1;31m' # red
-G = '\033[1;32m' # green
-O = '\033[1;33m' # orange
-B = '\033[1;34m' # blue
-P = '\033[1;35m' # purple
-C = '\033[1;36m' # cyan
-GR = '\033[1;37m' # gray
-T = '\033[1;93m' # tan
-###############################
diff --git a/core/Enumeration/scanenum.py b/core/Enumeration/scanenum.py
deleted file mode 100644
index ad3fa9ab..00000000
--- a/core/Enumeration/scanenum.py
+++ /dev/null
@@ -1,155 +0,0 @@
-#!/usr/bin/env python2
-# -*- coding: utf-8 -*-
-
-#-:-:-:-:-:-:-:-:-:-:-:-:#
-# TIDoS Framework #
-#-:-:-:-:-:-:-:-:-:-:-:-:#
-
-#Author: @_tID
-#This script is a part of TIDoS Framework
-#https://github.com/theInfectedDrake/TIDoS-Framework
-
-from __future__ import print_function
-import sys
-import time
-import os
-sys.path.append('modules/0x02-Scanning+Enumeration/')
-
-from nmapmain import *
-from colors import *
-from scanenumban import *
-from webscan import *
-from bannergrab import *
-from osdetect import *
-from webtech import *
-from waf import *
-from ssltlsscan import *
-from Crawling.crawlers import *
-from PortScans.portscan import *
-
-def scanenum(web):
-
- print(B+' [+] Module Selected : '+C+'Scanning and Enumeation')
- scanenumban()
- v = raw_input(''+O+' \033[4mTID\033[1;0m '+GR+':> ' + color.END)
- print('\n')
- if v.strip() == '1':
- print(B+' [!] Type Selected :'+C+' WAF Analysis'+O)
- waf(web)
- raw_input(O+' [#] Press '+GR+'Enter'+O+' to continue...')
- scanenum(web)
-
- elif v.strip() == '2':
- print(B+' [!] Type Selected :'+C+' Port Scanning')
- portscan(web)
- raw_input(O+' [#] Press '+GR+'Enter'+O+' to continue...')
- scanenum(web)
-
- elif v.strip() == '3':
- print(B+' [!] Type Selected :'+C+' Interactive NMap')
- nmapmain(web)
- raw_input(O+' [#] Press '+GR+'Enter'+O+' to continue...')
- scanenum(web)
-
- elif v.strip() == '4':
- print(B+' [!] Type Selected :'+C+' WebTech Fingerprinting')
- webtech(web)
- raw_input(O+' [#] Press '+GR+'Enter'+O+' to continue...')
- scanenum(web)
-
- elif v.strip() == '5':
- print(B+' [!] Type Selected :'+C+' SSL Enumeration')
- ssltlsscan(web)
- raw_input(O+' [#] Press '+GR+'Enter'+O+' to continue...')
- scanenum(web)
-
- elif v.strip() == '6':
- print(B+' [!] Type Selected :'+C+' OS Fingerprinting')
- osdetect(web)
- raw_input(O+' [#] Press '+GR+'Enter'+O+' to continue...')
- scanenum(web)
-
- elif v.strip() == '7':
- print(B+' [!] Type Selected :'+C+' Banner Grab')
- bannergrab(web)
- raw_input(O+' [#] Press '+GR+'Enter'+O+' to continue...')
- scanenum(web)
-
- elif v.strip() == '8':
- print(B+' [!] Type Selected :'+C+' IP Crawler')
- webscan(web)
- raw_input(O+' [#] Press '+GR+'Enter'+O+' to continue...')
- scanenumban()
- scanenum(web)
-
- elif v.strip() == '9':
- print(B+' [!] Type Selected :'+C+' Web Crawlers')
- crawlers(web)
- raw_input(O+' [#] Press '+GR+'Enter'+O+' to continue...')
- scanenum(web)
-
- elif v.strip() == 'A':
- print(' [!] Type Selected : All Modules')
- time.sleep(0.5)
-
- print(B+' [*] Firing up module -->'+C+' WAF Analysis'+O)
- waf(web)
- print(B+' [!] Module Completed -->'+C+' WAF Analysis\n')
- time.sleep(1)
-
- print(B+' [*] Firing up module -->'+C+' Port Scanning ')
- portscan(web)
- print(B+' [!] Module Completed -->'+C+' Port Scanning \n')
- time.sleep(1)
-
- print(B+' [*] Firing up module -->'+C+' Interactive NMap')
- nmapmain(web)
- print(B+' [!] Module Completed -->'+C+' NMap\n')
- time.sleep(1)
-
- print(B + ' [*] Firing up module -->' + C + ' WebTech Fingerprinting')
- webtech(web)
- print(B + ' [!] Module Completed -->' + C + ' WebTech\n')
- time.sleep(1)
-
- print(B + ' [*] Firing up module -->' + C + ' SSL Enumeration')
- ssltlsscan(web)
- print(B + ' [!] Module Completed -->' + C + ' SSL Enumeration\n')
- time.sleep(1)
-
- print(B + ' [*] Firing up module -->' + C + ' OS Detect')
- osdetect(web)
- print(B + ' [!] Module Completed -->' + C + ' OS Detect\n')
- time.sleep(1)
-
- print(B + ' [*] Firing up module -->' + C + ' Banner Grabbing')
- bannergrab(web)
- print(B + ' [!] Module Completed -->' + C + ' Banner Grabbing\n')
- time.sleep(1)
-
- print(B + ' [*] Firing up module -->' + C + ' Webserver Scanning')
- webscan(web)
- print(B + ' [!] Module Completed -->' + C + ' Webserver Scanning\n')
- time.sleep(1)
-
- print(B+' [*] Firing up module -->'+C+' Crawlers')
- crawlers(web)
- print(B+' [!] Module Completed -->'+C+' Crawlers\n')
- time.sleep(0.5)
-
- print(G+' [+] All modules successfully completed!')
- time.sleep(0.5)
- raw_input(O+' [#] Press '+GR+'Enter'+O+' to continue...')
- print(GR+' [*] Going back...')
-
- elif v.strip() == '99':
- print('[!] Back')
- time.sleep(0.7)
-
- else:
- dope = ['You high dude?','Hey there! Enter a valid option','Whoops! Thats not an option','Sorry fam! You just typed shit']
- print(dope[randint(0,3)])
- time.sleep(0.7)
- os.system('clear')
- scanenum(web)
-
diff --git a/core/Enumeration/scanenumban.py b/core/Enumeration/scanenumban.py
deleted file mode 100644
index d8580093..00000000
--- a/core/Enumeration/scanenumban.py
+++ /dev/null
@@ -1,37 +0,0 @@
-#!/usr/bin/env python2
-# -*- coding: utf-8 -*-
-
-#-:-:-:-:-:-:-:-:-:-:-:-:#
-# TIDoS Framework #
-#-:-:-:-:-:-:-:-:-:-:-:-:#
-
-#This script is a part of TIDoS Framework
-#https://github.com/theInfectedDrake/TIDoS-Framework
-
-from __future__ import print_function
-import os
-import time
-from core.Core.arts import scanenumban_art
-
-def scanenumban():
-
- os.system('clear')
- time.sleep(0.7)
- print(scanenumban_art)
- print("""
- Choose from the following options:
-
-\033[1;34m [1] \033[1;36mRemote Server WAF Enumeration \033[0m(Generic) (54 WAFs)
-\033[1;34m [2] \033[1;36mPort Scanning and Analysis \033[0m(Several Types)
-\033[1;34m [3] \033[1;36mInteractive Scanning with NMap \033[0m(16 Preloaded modules)
-\033[1;34m [4] \033[1;36mWeb Technologies Enumeration\033[0m(FrontEnd Technologies)
-\033[1;34m [5] \033[1;36mRemote Server SSL Enumeration\033[0m(Absolute)
-\033[1;34m [6] \033[1;36mOperating System Enumeration \033[0m(Absolute)
-\033[1;34m [7] \033[1;36mGrab Banners on Services \033[0m(via Open Ports)
-\033[1;34m [8] \033[1;36mScan all IP Addresses Linked to Domain \033[0m(CENSYS)
-\033[1;34m [9] \033[1;36mLet loose Crawlers on the target \033[0m(Depth 1, 2 & 3)
-
-\033[1;34m [A] \033[1;36mAutomate all one by one on target
-
-\033[1;34m [99] \033[1;36mBack
-""")
diff --git a/core/Exploitation/__init__.py b/core/Exploitation/__init__.py
deleted file mode 100644
index 73a0d197..00000000
--- a/core/Exploitation/__init__.py
+++ /dev/null
@@ -1,11 +0,0 @@
-#!/usr/bin/env python2
-# coding: utf-8
-
-#-:-:-:-:-:-:-:-:-:-:-:-:#
-# TIDoS Framework #
-#-:-:-:-:-:-:-:-:-:-:-:-:#
-
-#This module requires TIDoS Framework
-#https://github.com/theInfectedDrake/TIDoS-Framework
-
-pass
diff --git a/core/Exploitation/colors.py b/core/Exploitation/colors.py
deleted file mode 100644
index 96ed4433..00000000
--- a/core/Exploitation/colors.py
+++ /dev/null
@@ -1,37 +0,0 @@
-#!/usr/bin/env python2
-
-#-:-:-:-:-:-:-:-:-:-:-:-:#
-# TIDoS Framework #
-#-:-:-:-:-:-:-:-:-:-:-:-:#
-
-#This module requires TIDoS Framework
-#https://github.com/theInfectedDrake/TIDoS-Framework
-
-###############################
-class color:
- PURPLE = '\033[95m'
- CYAN = '\033[96m'
- DARKCYAN = '\033[36m'
- BLUE = '\033[94m'
- GREEN = '\033[92m'
- YELLOW = '\033[93m'
- RED = '\033[91m'
- BOLD = '\033[1m'
- UNDERLINE = '\033[4m'
- END = '\033[0m'
- HEADER = '\033[95m'
- OKBLUE = '\033[94m'
- OKGREEN = '\033[92m'
- WARNING = '\033[93m'
- FAIL = '\033[91m'
-
-W = '\033[1;0m' # white (normal)
-R = '\033[1;31m' # red
-G = '\033[1;32m' # green
-O = '\033[1;33m' # orange
-B = '\033[1;34m' # blue
-P = '\033[1;35m' # purple
-C = '\033[1;36m' # cyan
-GR = '\033[1;37m' # gray
-T = '\033[1;93m' # tan
-###############################
diff --git a/core/Exploitation/exploits.py b/core/Exploitation/exploits.py
deleted file mode 100644
index 4d0f6b3e..00000000
--- a/core/Exploitation/exploits.py
+++ /dev/null
@@ -1,36 +0,0 @@
-#!/usr/bin/env python2
-# -*- coding: utf-8 -*-
-
-#-:-:-:-:-:-:-:-:-:-:-:-:#
-# TIDoS Framework #
-#-:-:-:-:-:-:-:-:-:-:-:-:#
-
-#This module requires TIDoS Framework
-#https://github.com/theInfectedDrake/TIDoS-Framework
-
-from __future__ import print_function
-import sys
-import time
-import os
-from exploitsban import *
-from colors import *
-sys.path.append('modules/0x04-Exploitation+Loot/')
-
-from shellshock_exp import *
-
-def exploits(web):
-
- print('')
- print(B+' [+] Module Selected : '+C+'Exploitation')
- exploitsban()
- time.sleep(0.3)
- v = raw_input(GR+' [#] \033[1;4mTID\033[0m'+GR+' :> ' + color.END)
- print('')
- if v == '1':
- print(' [!] Type Selected : Shellshock Exploit')
- shellshock_exp(web)
- print(O+' [!] Completed!')
- time.sleep(1)
- print('\n\n')
- os.system('clear')
-
diff --git a/core/Exploitation/exploitsban.py b/core/Exploitation/exploitsban.py
deleted file mode 100644
index 46411295..00000000
--- a/core/Exploitation/exploitsban.py
+++ /dev/null
@@ -1,25 +0,0 @@
-#!/usr/bin/env python2
-# coding: utf-8
-
-#-:-:-:-:-:-:-:-:-:-:-:-:#
-# TIDoS Framework #
-#-:-:-:-:-:-:-:-:-:-:-:-:#
-
-#This module requires TIDoS Framework
-#https://github.com/theInfectedDrake/TIDoS-Framework
-
-from __future__ import print_function
-import os, time
-from core.Core.arts import exploitsban_art
-
-def exploitsban():
-
- os.system('clear')
- print(" [!] Module Selected : Exploits\n\n")
- time.sleep(0.5)
- print(exploitsban_art)
- print('''
-\033[1;36m [1] \033[1;34mShellShock Exploit\n
-\033[1;36m [!info] \033[1;34m This module is still being worked on...
-''')
-
diff --git a/core/Footprinting/Active_Recon/__init__.py b/core/Footprinting/Active_Recon/__init__.py
deleted file mode 100644
index 73a0d197..00000000
--- a/core/Footprinting/Active_Recon/__init__.py
+++ /dev/null
@@ -1,11 +0,0 @@
-#!/usr/bin/env python2
-# coding: utf-8
-
-#-:-:-:-:-:-:-:-:-:-:-:-:#
-# TIDoS Framework #
-#-:-:-:-:-:-:-:-:-:-:-:-:#
-
-#This module requires TIDoS Framework
-#https://github.com/theInfectedDrake/TIDoS-Framework
-
-pass
diff --git a/core/Footprinting/Active_Recon/activeban.py b/core/Footprinting/Active_Recon/activeban.py
deleted file mode 100644
index 421ae084..00000000
--- a/core/Footprinting/Active_Recon/activeban.py
+++ /dev/null
@@ -1,56 +0,0 @@
-#!/usr/bin/env python2
-# coding: utf-8
-
-#-:-:-:-:-:-:-:-:-:-:-:-:#
-# TIDoS Framework #
-#-:-:-:-:-:-:-:-:-:-:-:-:#
-
-#This module requires TIDoS Framework
-#https://github.com/theInfectedDrake/TIDoS-Framework
-
-from __future__ import print_function
-from colors import *
-import time
-
-def activeban():
-
- time.sleep(0.4)
- print(G+' +----------------+')
- print(G+' | '+O+'ACTIVE RECON'+G+' |')
- print(G+' +----------------+')
- time.sleep(0.3)
- print('')
- print(B+' [1]'+C+' Ping/NPing Enumeration'+W+' (Adaptative+Debug)')
- time.sleep(0.04)
- print(B+' [2]'+C+' Grab HTTP Headers'+W+' (Live Capture)')
- time.sleep(0.04)
- print(B+' [3]'+C+' Find Allowed HTTP Methods'+W+' (Via OPTIONS)')
- time.sleep(0.04)
- print(B+' [4]'+C+' Examine robots.txt and sitemap.xml')
- time.sleep(0.04)
- print(B+' [5]'+C+' Scrape Comments from Webpage'+W+' (Regex Based)')
- time.sleep(0.04)
- print(B+' [6]'+C+' Perform Advanced Traceroute'+W+' (TTL Based)')
- time.sleep(0.04)
- print(B+' [7]'+C+' Find Shared DNS Hosts'+W+' (NameServer Based)')
- time.sleep(0.04)
- print(B+' [8]'+C+' Examine SSL Certificate'+W+' (Absolute)')
- time.sleep(0.04)
- print(B+' [9]'+C+' CMS Detection '+W+'(185+ CMSs supported)')
- time.sleep(0.04)
- print(B+' [10]'+C+' Apache Status Disclosure'+W+' (File Based)')
- time.sleep(0.04)
- print(B+' [11]'+C+' WebDAV HTTP Enumeration'+W+' (SEARCH, PROFIND)')
- time.sleep(0.04)
- print(B+' [12]'+C+' Find PHPInfo File'+W+' (Regular Bruteforce)')
- time.sleep(0.04)
- print(B+' [13]'+C+' Enumerate Server behind website'+W)
- time.sleep(0.04)
- print(B+' [14]'+C+' Alternate Sites'+W+' (User-Agent Based)')
- time.sleep(0.04)
- print(B+' [15]'+C+' Common File Bruteforce'+W+' (5 modules)\n')
- time.sleep(0.04)
- print(B+' [A]'+C+' The Auto-Awesome Module\n')
- time.sleep(0.04)
- print(B+' [99]'+C+' Back\n')
-
diff --git a/core/Footprinting/Active_Recon/activeo.py b/core/Footprinting/Active_Recon/activeo.py
deleted file mode 100644
index 466c943f..00000000
--- a/core/Footprinting/Active_Recon/activeo.py
+++ /dev/null
@@ -1,246 +0,0 @@
-#!/usr/bin/env python2
-# coding: utf-8
-
-#-:-:-:-:-:-:-:-:-:-:-:-:#
-# TIDoS Framework #
-#-:-:-:-:-:-:-:-:-:-:-:-:#
-
-#This module requires TIDoS Framework
-#https://github.com/theInfectedDrake/TIDoS-Framework
-
-from __future__ import print_function
-import sys
-import os
-import time
-import subprocess
-import random
-from random import randint
-sys.path.append('modules/0x01-OSINT+Footprinting/0x02-ActiveReconnaissance/')
-
-from piwebenum import *
-from grabhead import *
-from httpmethods import *
-from robot import *
-from apachestat import *
-from dav import *
-from sharedns import *
-from commentssrc import *
-from sslcert import *
-from activeban import *
-from filebrute import *
-from traceroute import *
-from phpinfo import *
-from cms import *
-from serverdetect import *
-from altsites import *
-from colors import *
-
-def activeo(web):
-
- print(" [!] Module Selected : Active Reconnaissance\n\n")
- activeban()
- print('')
- time.sleep(0.3)
- v = raw_input (''+GR+' [#] \033[1;4mTID\033[0m'+GR+' :> ' + color.END)
- print('')
- if v.strip() == '1':
- print(C+' [!] Type Selected : Ping/NPing Enumeration')
- piwebenum(web)
- print('\n\n')
- raw_input(O+' [#] Press '+GR+'Enter'+O+' to continue...')
- activeo(web)
-
- elif v.strip() == '2':
- print(C+' [!] Type Selected : Grab HTTP Headers')
- grabhead(web)
- print('\n\n')
- raw_input(O+' [#] Press '+GR+'Enter'+O+' to continue...')
- activeo(web)
-
- elif v.strip() == '3':
- print(C+' [!] Type Selected : HTTP Allowed Methods')
- httpmethods(web)
- print('\n\n')
- raw_input(O+' [#] Press '+GR+'Enter'+O+' to continue...')
- activeo(web)
-
- elif v.strip() == '4':
- print(C+' [!] Type Selected : robots.txt and sitemap.xml Hunt')
- robot(web)
- print('\n\n')
- raw_input(O+' [#] Press '+GR+'Enter'+O+' to continue...')
- activeo(web)
-
- elif v.strip() == '5':
- print(C+' [!] Type Selected : Scrape Comments')
- commentssrc(web)
- print('\n\n')
- raw_input(O+' [#] Press '+GR+'Enter'+O+' to continue...')
- activeo(web)
-
- elif v.strip() == '6':
- print(C+' [!] Type Selected '+B+': Traceroute')
- traceroute(web)
- print('\n\n')
- raw_input(O+' [#] Press '+GR+'Enter'+O+' to continue...')
- activeo(web)
-
- elif v.strip() == '7':
- print(C+' [!] Type Selected : DNS Hosts')
- sharedns(web)
- print('\n\n')
- raw_input(O+' [#] Press '+GR+'Enter'+O+' to continue...')
- activeo(web)
-
- elif v.strip() == '8':
- print(C+' [!] Type Selected : SSL Certificate')
- sslcert(web)
- print('\n\n')
- raw_input(O+' [#] Press '+GR+'Enter'+O+' to continue...')
- activeo(web)
-
- elif v.strip() == '9':
- print(C+' [!] Type Selected : CMS Detection')
- cms(web)
- print('\n\n')
- raw_input(O+' [#] Press '+GR+'Enter'+O+' to continue...')
- activeo(web)
-
- elif v.strip() == '10':
- print(C+' [!] Type Selected : Apache Status')
- apachestat(web)
- print('\n\n')
- raw_input(O+' [#] Press '+GR+'Enter'+O+' to continue...')
- activeo(web)
-
- elif v.strip() == '11':
- print(C+' [!] Type Selected : WebDAV HTTP Enumeration')
- dav(web)
- print('\n\n')
- raw_input(O+' [#] Press '+GR+'Enter'+O+' to continue...')
- activeo(web)
-
- elif v.strip() == '12':
- print(C+' [!] Type Selected : PHPInfo Enumeration')
- phpinfo(web)
- print('\n\n')
- raw_input(O+' [#] Press '+GR+'Enter'+O+' to continue...')
- activeo(web)
-
- elif v.strip() == '13':
- print(C+' [!] Type Selected : Server Detection')
- serverdetect(web)
- print('\n\n')
- raw_input(O+' [#] Press '+GR+'Enter'+O+' to continue...')
- activeo(web)
-
- elif v.strip() == '14':
- print(C+' [!] Type Selected : Alternate Sites ')
- altsites(web)
- print('\n\n')
- raw_input(O+' [#] Press '+GR+'Enter'+O+' to continue...')
- activeo(web)
-
- elif v.strip() == '15':
- print(C+' [!] Type Selected : File Bruteforcers')
- filebrute(web)
- print('\n\n')
- raw_input(O+' [#] Press '+GR+'Enter'+O+' to continue...')
- activeo(web)
-
- elif v.strip() == 'A':
- print(C+' [!] Type Selected : All Modules')
- time.sleep(0.5)
- print(C+' [*] Firing up module -->'+B+' Ping Enum')
- piwebenum(web)
- print(C+' [!] Module Completed -->'+B+' PIng\n')
-
- time.sleep(1)
- print(C+' [*] Firing up module -->'+B+' Grab Headers')
- grabhead(web)
- print(C+'\n [!] Module Completed -->'+B+' Grabhead\n')
-
- time.sleep(1)
- print(C+' [*] Firing up module -->'+B+' Robots.txt Hunter')
- robot(web)
- print(C+'\n [!] Module Completed -->'+B+' Robot Hunter\n')
- time.sleep(1)
-
- print(C+' [*] Firing up module -->'+B+' Comments Scraper')
- commentssrc(web)
- print(C+'\n [!] Module Completed -->'+B+' Comments Src\n')
- time.sleep(1)
-
- print(C+' [*] Firing up module -->'+B+' Traceroute')
- traceroute(web)
- print(C+'\n [!] Module Completed -->'+B+' Traceroute\n')
- time.sleep(1)
-
- print(C+' [*] Firing up module -->'+B+' Shared DNS Servers')
- sharedns(web)
- print(C+'\n [!] Module Completed -->'+B+' Shared DNS Servers\n')
- time.sleep(1)
-
- print(C+' [*] Firing up module -->'+B+' SSl Certificate Info')
- sslcert(web)
- print(C+'\n [!] Module Completed -->'+B+' SSl Cert\n')
- time.sleep(1)
-
- print(C+' [*] Firing up module -->'+B+' CMS Detection')
- cms(web)
- print(C+'\n [!] Module Completed -->'+B+' CMS Detect\n')
- time.sleep(1)
-
- print(C+' [*] Firing up module -->'+B+' WebDAV HTTP Profiling')
- dav(web)
- print(C+'\n [!] Module Completed -->'+B+' WebDAV HTTP Profiling\n')
- time.sleep(1)
-
- print(C+' [*] Firing up module -->'+B+' Apache Status')
- apachestat(web)
- print(C+'\n [!] Module Completed -->'+B+' Apache Status\n')
- time.sleep(1)
-
- print(C+' [*] Firing up module -->'+B+' PHPInfo')
- phpinfo(web)
- print(C+'\n [!] Module Completed -->'+B+' PHPInfo\n')
- time.sleep(1)
-
- print(C+' [*] Firing up module -->'+B+' OS Fingerprinting')
- osdetect(web)
- print(C+'\n [!] Module Completed -->'+B+' OS Detect\n')
- time.sleep(1)
-
- print(C+' [*] Firing up module -->'+B+' File bruteforcer')
- filebrute(web)
- print(C+'\n [!] Module Completed -->'+B+' File Bruteforcer\n')
- time.sleep(1)
-
- print(C+' [*] Firing up module -->'+B+' Server Detection')
- serverdetect(web)
- print(C+'\n [!] Module Completed -->'+B+' Server Detect\n')
- time.sleep(1)
-
- print(C+' [*] Firing up module -->'+B+' Alt. Sites')
- altsites(web)
- print(C+'\n [!] Module Completed -->'+B+' Alt. Sites\n')
- time.sleep(1)
-
- print(C+'\n [!] All scantypes have been tested on target...')
- raw_input(O+' [#] Press '+GR+'Enter'+O+' to continue...')
- print(C+' [*] Going back to menu...')
- time.sleep(3)
- os.system('clear')
- activeo(web)
-
- elif v.strip() == '99':
- print(C+' [*] Back to the menu !')
- os.system('clear')
-
- else:
- dope = ['You high dude?','Shit! Enter a valid option','Whoops! Thats not an option','Sorry! You just typed shit']
- print(dope[randint(0,3)])
- time.sleep(0.7)
- os.system('clear')
- activeo(web)
-
diff --git a/core/Footprinting/Info_Disc/__init__.py b/core/Footprinting/Info_Disc/__init__.py
deleted file mode 100644
index 73a0d197..00000000
--- a/core/Footprinting/Info_Disc/__init__.py
+++ /dev/null
@@ -1,11 +0,0 @@
-#!/usr/bin/env python2
-# coding: utf-8
-
-#-:-:-:-:-:-:-:-:-:-:-:-:#
-# TIDoS Framework #
-#-:-:-:-:-:-:-:-:-:-:-:-:#
-
-#This module requires TIDoS Framework
-#https://github.com/theInfectedDrake/TIDoS-Framework
-
-pass
diff --git a/core/Footprinting/Info_Disc/infodisc.py b/core/Footprinting/Info_Disc/infodisc.py
deleted file mode 100644
index 1ceec3c5..00000000
--- a/core/Footprinting/Info_Disc/infodisc.py
+++ /dev/null
@@ -1,124 +0,0 @@
-#!/usr/bin/env python2
-# coding:'+B+' utf-8
-
-#-:'+B+'-:-:-:-:-:-:-:-:-:-:-:#
-# TIDoS Framework #
-#-:-:-:-:-:-:-:-:-:-:-:-:#
-
-#This module requires TIDoS Framework
-#https://github.com/theInfectedDrake/TIDoS-Framework
-
-from __future__ import print_function
-import sys
-import os
-import time
-import subprocess
-import random
-from random import randint
-from subprocess import call
-sys.path.append('modules/0x01-OSINT+Footprinting/0x03-InformationDisclosure/')
-
-from credit import *
-from emailext import *
-from errors import *
-from phone import *
-from ssn import *
-from infodiscban import *
-from colors import *
-from internalip import *
-
-def infodisc(web):
-
- print(" [!] Module Selected : Information Disclosure\n\n")
- infodiscban()
- print('')
- time.sleep(0.3)
- v = raw_input (''+GR+' [#] \033[1;4mTID\033[0m'+GR+' :> ' + color.END)
- print('')
- if v == '1':
- print(C+' [!] Type Selected :'+B+' Credit Card Enumeration')
- credit(web)
- print('\n\n')
- raw_input(O+' [#] Press '+GR+'Enter'+O+' to continue...')
- infodisc(web)
-
- elif v == '2':
- print(C+' [!] Type Selected :'+B+' Extract All Emails')
- emailext(web)
- print('\n\n')
- raw_input(O+' [#] Press '+GR+'Enter'+O+' to continue...')
- infodisc(web)
-
- elif v == '3':
- print(C+' [!] Type Selected :'+B+' Enumerate Errors + FPD')
- errors(web)
- print('\n\n')
- raw_input(O+' [#] Press '+GR+'Enter'+O+' to continue...')
- infodisc(web)
-
- elif v == '4':
- print(C+' [!] Type Selected :'+B+' Internal IP disclosure')
- internalip(web)
- print('\n\n')
- raw_input(O+' [#] Press '+GR+'Enter'+O+' to continue...')
- infodisc(web)
-
- elif v == '5':
- print(C+' [!] Type Selected :'+B+' Phone Numbers Extract')
- phone(web)
- print('\n\n')
- raw_input(O+' [#] Press '+GR+'Enter'+O+' to continue...')
- infodisc(web)
-
- elif v == '6':
- print(C+' [!] Type Selected :'+B+' Social Security Numbers')
- ssn(web)
- print('\n\n')
- raw_input(O+' [#] Press '+GR+'Enter'+O+' to continue...')
- infodisc(web)
-
- elif v == 'A':
- print(C+' [!] Type Selected :'+B+' All Modules')
- time.sleep(0.5)
- print(C+' [*] Firing up module -->'+B+' Credit Cards')
- credit(web)
- print(C+' [!] Module Completed -->'+B+' Credit Cards\n')
-
- time.sleep(1)
- print(C+' [*] Firing up module -->'+B+' Email Extraction')
- emailext(web)
- print(C+' [!] Module Completed -->'+B+' Email Hunt\n')
-
- time.sleep(1)
- print(C+' [*] Firing up module -->'+B+' Errors Enumeration + FPD')
- errors(web)
- print(C+' [!] Module Completed -->'+B+' Errors Enumeration\n')
- time.sleep(1)
-
- print(C+' [*] Firing up module -->'+B+' Extract Phone Numbers')
- phone(web)
- print(C+' [!] Module Completed -->'+B+' Extract Phone Numbers\n')
- time.sleep(1)
-
- print(C+' [*] Firing up module -->'+B+' Extract Social Security Numbers')
- ssn(web)
- print(C+' [!] Module Completed -->'+B+' Extract SSN\n')
- time.sleep(1)
-
- print(C+' [!] All scantypes have been tested on target...')
- time.sleep(1)
- print(C+' [*] Going back to menu...')
- infodisc(web)
-
- elif v == '99':
- print(C+' [*] Back to the menu !')
- time.sleep(0.8)
- os.system('clear')
-
- else:
- dope = ['You high dude?','Shit! Enter a valid option','Whoops! Thats not an option','Sorry! You just typed shit']
- print(dope[randint(0,3)])
- time.sleep(0.7)
- os.system('clear')
- infodisc(web)
-
diff --git a/core/Footprinting/Info_Disc/infodiscban.py b/core/Footprinting/Info_Disc/infodiscban.py
deleted file mode 100644
index c625b2b4..00000000
--- a/core/Footprinting/Info_Disc/infodiscban.py
+++ /dev/null
@@ -1,38 +0,0 @@
-#!/usr/bin/env python2
-# coding: utf-8
-
-#-:-:-:-:-:-:-:-:-:-:-:-:#
-# TIDoS Framework #
-#-:-:-:-:-:-:-:-:-:-:-:-:#
-
-#This module requires TIDoS Framework
-#https://github.com/theInfectedDrake/TIDoS-Framework
-
-from __future__ import print_function
-from colors import *
-import time
-
-def infodiscban():
-
- time.sleep(0.4)
- print(G+'\n +-------------------+')
- print(G+' | '+O+'INFO DISCLOSURE'+G+' |')
- print(G+' +-------------------+')
- time.sleep(0.3)
- print('')
- print(B+' [1]'+C+' Find out Credit Cards '+W+'(if disclosed in plain text)')
- time.sleep(0.1)
- print(B+' [2]'+C+' Extract out all existing emails'+W+' (Absolute)')
- time.sleep(0.1)
- print(B+' [3]'+C+' Enumerate Errors '+W+'(Includes Full Path Disclosure)')
- time.sleep(0.1)
- print(B+' [4]'+C+' Find out any leaks of internal IP addresses')
- time.sleep(0.1)
- print(B+' [5]'+C+' Extract out all Phone Numbers '+W+'(if plaintext disclosure)')
- time.sleep(0.1)
- print(B+' [6]'+C+' Extract out all Social Security Numbers '+W+'(US Based)\n')
- time.sleep(0.1)
- print(B+' [A]'+C+' Start up ALL modules 1 by 1\n')
- time.sleep(0.1)
- print(B+' [99]'+C+' Back\n')
-
diff --git a/core/Footprinting/Passive_Recon/__init__.py b/core/Footprinting/Passive_Recon/__init__.py
deleted file mode 100644
index 73a0d197..00000000
--- a/core/Footprinting/Passive_Recon/__init__.py
+++ /dev/null
@@ -1,11 +0,0 @@
-#!/usr/bin/env python2
-# coding: utf-8
-
-#-:-:-:-:-:-:-:-:-:-:-:-:#
-# TIDoS Framework #
-#-:-:-:-:-:-:-:-:-:-:-:-:#
-
-#This module requires TIDoS Framework
-#https://github.com/theInfectedDrake/TIDoS-Framework
-
-pass
diff --git a/core/Footprinting/Passive_Recon/passiveban.py b/core/Footprinting/Passive_Recon/passiveban.py
deleted file mode 100644
index 190ac7c6..00000000
--- a/core/Footprinting/Passive_Recon/passiveban.py
+++ /dev/null
@@ -1,74 +0,0 @@
-#!/usr/bin/env python2
-# -*- coding: utf-8 -*-
-
-#-:-:-:-:-:-:-:-:-:-:-:-:#
-# TIDoS Framework #
-#-:-:-:-:-:-:-:-:-:-:-:-:#
-
-#Author: @_tID
-#This module requires TIDoS Framework
-#https://github.com/theInfectedDrake/TIDoS-Framework
-
-from __future__ import print_function
-import time
-from colors import *
-
-def passiveban():
-
- time.sleep(0.4)
- print(G+' +-----------------+')
- print(G+' | '+O+'PASSIVE RECON '+G+'|')
- print(G+' +-----------------+')
- time.sleep(0.6)
- print('')
- print(B+' [1] '+C+'Ping Check '+W+'(Using external APi)')
- time.sleep(0.04)
- print(B+' [2] '+C+'WhoIS Lookup '+W+'(Get domain info)')
- time.sleep(0.04)
- print(B+' [3] '+C+'GeoIP Lookup '+W+'(Pinpoint Server Location)')
- time.sleep(0.04)
- print(B+' [4]'+C+' DNS Configuration Lookup'+W+' (DNSDump)')
- time.sleep(0.04)
- print(B+' [5]'+C+' Gather Subdomains '+W+'(Only indexed ones)')
- time.sleep(0.04)
- print(B+' [6]'+C+' Reverse DNS Configuration Lookup ')
- time.sleep(0.04)
- print(B+' [7]'+C+' Subnet Enumeration '+W+'(Class Based)')
- time.sleep(0.04)
- print(B+' [8]'+C+' Reverse IP Lookup '+W+'(Hosts on same server)')
- time.sleep(0.04)
- print(B+' [9]'+C+' Domain IP History '+W+'(IP History Instances)')
- time.sleep(0.04)
- print(B+' [10]'+C+' Gather All Links from WebPage '+W+'(Indexed ones)')
- time.sleep(0.04)
- print(B+' [11]'+C+' Google Search '+W+'(Search your own Query or Dork)')
- time.sleep(0.04)
- print(B+' [12] '+C+'Google Dorking '+W+'(Multiple Modules)')
- time.sleep(0.04)
- print(B+' [13] '+C+'Wayback Machine Lookup '+W+'(pure backups)')
- time.sleep(0.04)
- print(B+' [14] '+C+'Hacked Email Check '+W+'(Breached/leaked emails)')
- time.sleep(0.04)
- print(B+' [15] '+C+'Email to Domain Resolver '+W+'(Email whois)')
- time.sleep(0.04)
- print(B+' [16] '+C+'Email Enumeration via Google Groups '+W+'')
- time.sleep(0.04)
- print(B+' [17] '+C+'Check Alias Availability '+W+'(Social Networks)')
- time.sleep(0.04)
- print(B+' [18] '+C+'Find PasteBin Posts '+W+'(Domain Based)')
- time.sleep(0.04)
- print(B+' [19] '+C+'LinkedIn Gathering '+W+'(Employees, Companies)')
- time.sleep(0.04)
- print(B+' [20] '+C+'Google Plus Gathering '+W+'(Profiles Crawling)')
- time.sleep(0.04)
- print(B+' [21] '+C+'Public Contact Info Gathering '+W+'(Full Contact)')
- time.sleep(0.04)
- print(B+' [22] '+C+'CENSYS Domain Reconnaissance '+W+'(CENSYS.IO)')
- time.sleep(0.04)
- print(B+' [23] '+C+'Threat Intelligence Gathering '+W+'(Bad IPs)\n')
- time.sleep(0.04)
- print(B+' [A] '+C+'The Auto-Awesome Module '+W+'(Unleash the Beast)\n')
- time.sleep(0.04)
- print(B+' [99] '+C+'Back'+W+'\n')
- time.sleep(0.3)
-
diff --git a/core/Footprinting/Passive_Recon/passiveo.py b/core/Footprinting/Passive_Recon/passiveo.py
deleted file mode 100644
index 3966dd3c..00000000
--- a/core/Footprinting/Passive_Recon/passiveo.py
+++ /dev/null
@@ -1,324 +0,0 @@
-#!/usr/bin/env python2
-#-*- coding: utf-8 -*-
-
-#-:-:-:-:-:-:-:-:-:-:-:-:#
-# TIDoS Framework #
-#-:-:-:-:-:-:-:-:-:-:-:-:#
-
-#This module requires TIDoS Framework
-#https://github.com/theInfectedDrake/TIDoS-Framework
-
-from __future__ import print_function
-import sys
-import time
-import os
-from random import randint
-sys.path.append('modules/0x01-OSINT+Footprinting/0x01-PassiveReconnaissance/')
-
-from dnschk import *
-from passiveban import *
-from piweb import *
-from getgeoip import *
-from revip import *
-from revdns import *
-from getconinfo import *
-from subdom import *
-from googlenum import *
-from links import *
-from censysdom import *
-from subnet import *
-from hackedmail import *
-from iphistory import *
-from mailtodom import *
-from checkuser import *
-from googlegroups import *
-from threatintel import *
-from webarchive import *
-from googledorker import *
-from googleSearch import *
-from whoischeckup import *
-from pastebin import *
-from linkedin import *
-from colors import *
-
-def passiveo(web):
-
- print(" [!] Module Selected : Passive Reconnaissance\n")
- passiveban()
- v = raw_input (''+GR+' [#] \033[1;4mTID\033[0m'+GR+' :> ' + color.END)
- print('\n\n')
- if v.strip() == '1':
- print(C+' [!] Type Selected '+B+': Ping Check')
- piweb(web)
- raw_input(O+' [#] Press '+GR+'Enter'+O+' to continue...')
- passiveo(web)
-
- elif v.strip() == '2':
- print(C+' [!] Type Selected '+B+': WhoIS Checkup')
- whoischeckup(web)
- raw_input(O+' [#] Press '+GR+'Enter'+O+' to continue...')
- passiveo(web)
-
- elif v.strip() == '3':
- print(C+' [!] Type Selected '+B+': GeoIP Lookup')
- getgeoip(web)
- raw_input(O+' [#] Press '+GR+'Enter'+O+' to continue...')
- passiveo(web)
-
- elif v.strip() == '4':
- print(C+' [!] Type Selected '+B+': DNS Lookup')
- dnschk(web)
- raw_input(O+' [#] Press '+GR+'Enter'+O+' to continue...')
- passiveo(web)
-
- elif v.strip() == '5':
- print(C+' [!] Type Selected '+B+': Subdomain Scan')
- subdom(web)
- raw_input(O+' [#] Press '+GR+'Enter'+O+' to continue...')
- passiveo(web)
-
- elif v.strip() == '6':
- print(C+' [!] Type Selected '+B+': Reverse DNS Lookup')
- revdns(web)
- raw_input(O+' [#] Press '+GR+'Enter'+O+' to continue...')
- passiveo(web)
-
- elif v.strip() == '7':
- print(C+' [!] Type Selected : Subnet Enumeration')
- subnet(web)
- raw_input(O+' [#] Press '+GR+'Enter'+O+' to continue...')
- passiveo(web)
-
- elif v.strip() == '8':
- print(C+' [!] Type Selected '+B+': Reverse IP Lookup')
- revip(web)
- raw_input(O+' [#] Press '+GR+'Enter'+O+' to continue...')
- passiveo(web)
-
- elif v.strip() == '9':
- print(C+' [!] Type Selected '+B+': IP History')
- iphistory(web)
- raw_input(O+' [#] Press '+GR+'Enter'+O+' to continue...')
- passiveo(web)
-
- elif v.strip() == '10':
- print(C+' [!] Type Selected '+B+': Page Links')
- links(web)
- raw_input(O+' [#] Press '+GR+'Enter'+O+' to continue...')
- passiveo(web)
-
- elif v.strip() == '11':
- print(C+' [!] Type Selected '+B+': Google Search')
- googleSearch()
- raw_input(O+' [#] Press '+GR+'Enter'+O+' to continue...')
- passiveo(web)
-
- elif v.strip() == '12':
- print(C+' [!] Type Selected '+B+': Google Dorker')
- googledorker(web)
- raw_input(O+' [#] Press '+GR+'Enter'+O+' to continue...')
- passiveo(web)
-
- elif v.strip() == '13':
- print(C+' [!] Type Selected '+B+': Wayback Machine')
- webarchive(web)
- raw_input(O+' [#] Press '+GR+'Enter'+O+' to continue...')
- passiveo(web)
-
- elif v.strip() == '14':
- print(C+' [!] Type Selected '+B+': Hacked Email Check')
- hackedmail()
- raw_input(O+' [#] Press '+GR+'Enter'+O+' to continue...')
- passiveo(web)
-
- elif v.strip() == '15':
- print(C+' [!] Type Selected '+B+': Mail to Domain')
- mailtodom()
- raw_input(O+' [#] Press '+GR+'Enter'+O+' to continue...')
- passiveo(web)
-
- elif v.strip() == '16':
- print(C+' [!] Type Selected '+B+': Google Groups Enum')
- googlegroups(web)
- raw_input(O+' [#] Press '+GR+'Enter'+O+' to continue...')
- passiveo(web)
-
- elif v.strip() == '17':
- print(C+' [!] Type Selected '+B+': Check Username')
- checkuser(web)
- raw_input(O+' [#] Press '+GR+'Enter'+O+' to continue...')
- passiveo(web)
-
- elif v.strip() == '18':
- print(C+' [!] Type Selected '+B+': PasteBin Posts')
- pastebin(web)
- raw_input(O+' [#] Press '+GR+'Enter'+O+' to continue...')
- passiveo(web)
-
- elif v.strip() == '19':
- print(C+' [!] Type Selected '+B+': LinkedIn Gathering')
- linkedin(web)
- raw_input(O+' [#] Press '+GR+'Enter'+O+' to continue...')
- passiveo(web)
-
- elif v.strip() == '20':
- print(C+' [!] Type Selected '+B+': Google Plus Gathering')
- googlenum(web)
- raw_input(O+' [#] Press '+GR+'Enter'+O+' to continue...')
- passiveo(web)
-
- elif v.strip() == '21':
- print(C+' [!] Type Selected '+B+': Public Contact Info')
- getconinfo(web)
- raw_input(O+' [#] Press '+GR+'Enter'+O+' to continue...')
- passiveo(web)
-
- elif v.strip() == '22':
- print(C+' [!] Type Selected '+B+': CENSYS Gathering')
- censysdom(web)
- raw_input(O+' [#] Press '+GR+'Enter'+O+' to continue...')
- passiveo(web)
-
- elif v.strip() == '23':
- print(C+' [!] Type Selected '+B+': Threat Intel Gathering')
- threatintel(web)
- raw_input(O+' [#] Press '+GR+'Enter'+O+' to continue...')
- passiveo(web)
-
- elif v.strip() == '99':
- print(C+' [*] Back!')
- time.sleep(0.7)
- os.system('clear')
-
- elif v.strip() == 'A':
- print(C+' [!] Type Selected '+B+': All Modules\n')
- time.sleep(0.5)
-
- print(C+' [*] Firing up module -->'+O+' Ping Check')
- piweb(web)
- print(C+'\n [!] Module Completed -->'+O+' Piweb\n')
- time.sleep(1)
-
- print(C+' [*] Firing up module -->'+O+' WHOIS ')
- whoischeckup(web)
- print(C+'\n [!] Module Completed -->'+O+' WHOIS\n')
- time.sleep(1)
-
- print(C+' [*] Firing up module -->'+O+' GeoIP Lookup ')
- getgeoip(web)
- print(C+'\n [!] Module Completed -->'+O+' GeoIP\n')
- time.sleep(1)
-
- print(C+' [*] Firing up module -->'+O+' DNS Enumeration ')
- dnschk(web)
- print(C+'\n [!] Module Completed -->'+O+' DNS Enum\n')
- time.sleep(1)
-
- print(C+' [*] Firing up module -->'+O+' Subdomain Scan ')
- subdom(web)
- print(C+'\n [!] Module Completed -->'+O+' Subdom\n')
- time.sleep(1)
-
- print(C+' [*] Firing up module -->'+O+' Subnet Calc. ')
- subnet(web)
- print(C+'\n [!] Module Completed -->'+O+' Subnet Calculation\n')
- time.sleep(1)
-
- print(C+' [*] Firing up module -->'+O+' Reverse DNS Lookup ')
- revdns(web)
- print(C+'\n [!] Module Completed -->'+O+' RevDNS\n')
- time.sleep(1)
-
- print(C+' [*] Firing up module -->'+O+' IP History Lookup ')
- iphistory(web)
- print(C+'\n [!] Module Completed -->'+O+' RevIP\n')
- time.sleep(1)
-
- print(C+' [*] Firing up module -->'+O+' Reverse IP Lookup ')
- revip(web)
- print(C+'\n [!] Module Completed -->'+O+' RevIP\n')
- time.sleep(1)
-
- print(C+' [*] Firing up module -->'+O+' Web Links ')
- links(web)
- print(C+'\n [!] Module Completed -->'+O+' Web Links\n')
- time.sleep(1)
-
- print(C+' [!] Firing up module -->'+O+' Google Search')
- googleSearch()
- print(C+'\n [!] Module Completed -->'+O+' GSearch\n')
- time.sleep(1)
-
- print(C+' [!] Firing up module -->'+O+' WayBack Machine')
- webarchive(web)
- print(C+'\n [!] Module Completed -->'+O+' WayBack Machine\n')
- time.sleep(1)
-
- print(C+' [!] Firing up module -->'+O+' Hacked Email Checker')
- hackedmail()
- print(C+'\n [!] Module Completed -->'+O+' Hacked Email\n')
- time.sleep(1)
-
- print(C+' [!] Firing up module -->'+O+' Mail to Domain')
- mailtodom()
- print(C+'\n [!] Module Completed -->'+O+' Mail to Domain\n')
- time.sleep(1)
-
- print(C+' [*] Firing up module -->'+O+' Google Groups Enum')
- googlegroups(web)
- print(C+'\n [!] Module Completed -->'+O+' GGroups\n')
- time.sleep(1)
-
- print(C+' [*] Firing up module -->'+O+' Check Username')
- checkuser(web)
- print(C+'\n [!] Module Completed -->'+O+' Username\n')
- time.sleep(1)
-
- print(C+' [*] Firing up module -->'+O+' LinkedIn gathering')
- linkedin(web)
- print(C+'\n [!] Module Completed -->'+O+' LinkedIn\n')
- time.sleep(1)
-
- print(C+' [*] Firing up module -->'+O+' Pastebin Posts')
- pastebin(web)
- print(C+'\n [!] Module Completed -->'+O+' Pastebin\n')
- time.sleep(1)
-
- print(C+' [*] Firing up module -->'+O+' Google Plus Gathering')
- googlenum(web)
- print(C+'\n [!] Module Completed -->'+O+' Google Plus Gathering\n')
- time.sleep(1)
-
- print(C+' [*] Firing up module -->'+O+' Google Dorker')
- googledorker(web)
- print(C+'\n [!] Module Completed -->'+O+' GDorker\n')
- time.sleep(1)
-
- print(C+' [*] Firing up module -->'+O+' Public Contact Info')
- getconinfo(web)
- print(C+'\n [!] Module Completed -->'+O+' Public Contact info\n')
- time.sleep(1)
-
- print(C+' [*] Firing up module -->'+O+' Gathering via CENSYS')
- censysdom(web)
- print(C+'\n [!] Module Completed -->'+O+' CENSYS Gathering\n')
- time.sleep(1)
-
- print(C+' [*] Firing up module -->'+O+' Threat Intelligence')
- threatintel(web)
- print(C+'\n [!] Module Completed -->'+O+' Threat Intelligence\n')
- time.sleep(1)
-
- print(G+' [+] All modules successfully completed!')
- time.sleep(3)
- raw_input(O+' [#] Press '+GR+'Enter'+O+' to continue...')
- print(GR+' [*] Going back...')
- os.system('clear')
- passiveo(web)
-
- else:
- dope = ['You high dude?','Shit! Enter a valid option','Whoops! Thats not an option','Sorry! You just typed shit']
- print(C+'\n [!] '+dope[randint(0,3)])
- os.system('clear')
- passiveo(web)
-
diff --git a/core/Footprinting/__init__.py b/core/Footprinting/__init__.py
deleted file mode 100644
index 73a0d197..00000000
--- a/core/Footprinting/__init__.py
+++ /dev/null
@@ -1,11 +0,0 @@
-#!/usr/bin/env python2
-# coding: utf-8
-
-#-:-:-:-:-:-:-:-:-:-:-:-:#
-# TIDoS Framework #
-#-:-:-:-:-:-:-:-:-:-:-:-:#
-
-#This module requires TIDoS Framework
-#https://github.com/theInfectedDrake/TIDoS-Framework
-
-pass
diff --git a/core/Footprinting/colors.py b/core/Footprinting/colors.py
deleted file mode 100644
index 96ed4433..00000000
--- a/core/Footprinting/colors.py
+++ /dev/null
@@ -1,37 +0,0 @@
-#!/usr/bin/env python2
-
-#-:-:-:-:-:-:-:-:-:-:-:-:#
-# TIDoS Framework #
-#-:-:-:-:-:-:-:-:-:-:-:-:#
-
-#This module requires TIDoS Framework
-#https://github.com/theInfectedDrake/TIDoS-Framework
-
-###############################
-class color:
- PURPLE = '\033[95m'
- CYAN = '\033[96m'
- DARKCYAN = '\033[36m'
- BLUE = '\033[94m'
- GREEN = '\033[92m'
- YELLOW = '\033[93m'
- RED = '\033[91m'
- BOLD = '\033[1m'
- UNDERLINE = '\033[4m'
- END = '\033[0m'
- HEADER = '\033[95m'
- OKBLUE = '\033[94m'
- OKGREEN = '\033[92m'
- WARNING = '\033[93m'
- FAIL = '\033[91m'
-
-W = '\033[1;0m' # white (normal)
-R = '\033[1;31m' # red
-G = '\033[1;32m' # green
-O = '\033[1;33m' # orange
-B = '\033[1;34m' # blue
-P = '\033[1;35m' # purple
-C = '\033[1;36m' # cyan
-GR = '\033[1;37m' # gray
-T = '\033[1;93m' # tan
-###############################
diff --git a/core/Footprinting/footprint.py b/core/Footprinting/footprint.py
deleted file mode 100644
index eb99ee8f..00000000
--- a/core/Footprinting/footprint.py
+++ /dev/null
@@ -1,57 +0,0 @@
-#!/usr/bin/env python2
-# -*- coding: utf-8 -*-
-
-#-:-:-:-:-:-:-:-:-:-:-:-:#
-# TIDoS Framework #
-#-:-:-:-:-:-:-:-:-:-:-:-:#
-
-#This module requires TIDoS Framework
-#https://github.com/theInfectedDrake/TIDoS-Framework
-
-from __future__ import print_function
-import os
-import time
-
-# Get module imports
-from colors import *
-from footprintban import *
-from Info_Disc.infodisc import *
-from Active_Recon.activeo import *
-from Passive_Recon.passiveo import *
-
-def footprint(web):
-
- print(B+' [+] Module Selected : '+C+'Footprinting')
- footprintban() # banner
- m = raw_input(O+' [#] \033[1;4mTID\033[0m'+GR+' :> ' + color.END)
- print('')
-
- if m.strip() == '1':
- passiveo(web) # passive recon module
- print('\n')
- raw_input(O+' [#] Press '+GR+'Enter'+O+' to continue...')
- footprint(web)
-
- elif m.strip() == '2':
- activeo(web) # active recon module
- print('\n')
- raw_input(O+' [#] Press '+GR+'Enter'+O+' to continue...')
- footprint(web)
-
- elif m.strip() == '3':
- infodisc(web) # information disclosure modules
- print('\n')
- raw_input(O+' [#] Press '+GR+'Enter'+O+' to continue...')
- footprint(web)
-
- elif m.strip() == '99': # get back to main module
- print(' [+] Back!')
-
- else:
-
- print('')
- dope = [' [*] You high dude?',' [*] Hey there! Enter a valid option',' [*] Whoops! Thats not an option',' [*] Sorry fam! You just typed shit'] # troll stuff :p
- print(dope[randint(0,3)])
- time.sleep(0.5)
- footprint(web)
-
diff --git a/core/Footprinting/footprintban.py b/core/Footprinting/footprintban.py
deleted file mode 100644
index 37375b8c..00000000
--- a/core/Footprinting/footprintban.py
+++ /dev/null
@@ -1,30 +0,0 @@
-#!/usr/bin/env python2
-# coding: utf-8
-
-#-:-:-:-:-:-:-:-:-:-:-:-:#
-# TIDoS Framework #
-#-:-:-:-:-:-:-:-:-:-:-:-:#
-
-#This module requires TIDoS Framework
-#https://github.com/theInfectedDrake/TIDoS-Framework
-
-from __future__ import print_function
-import os, time
-from colors import *
-from core.Core.arts import footprintban_art
-
-def footprintban():
-
- time.sleep(0.5)
- os.system('clear')
- print(G+' [!] Module Loaded : Reconnaissance\n')
- print(footprintban_art)
- print("""
-Choose from the following options:
-
-\033[1;34m [1] \033[1;36mPassive Footprinting \033[1;0m(Open Source Intelligence)
-\033[1;34m [2] \033[1;36mActive Reconnaissance \033[1;0m(Gather via Interaction)
-\033[1;34m [3] \033[1;36mInformation Disclosure \033[1;0m(Errors, Emails, etc)
-
-\033[1;34m [99] \033[1;36mBack
-""")
diff --git a/core/Vulnlysis/Misc_Bugs/__init__.py b/core/Vulnlysis/Misc_Bugs/__init__.py
deleted file mode 100644
index 73a0d197..00000000
--- a/core/Vulnlysis/Misc_Bugs/__init__.py
+++ /dev/null
@@ -1,11 +0,0 @@
-#!/usr/bin/env python2
-# coding: utf-8
-
-#-:-:-:-:-:-:-:-:-:-:-:-:#
-# TIDoS Framework #
-#-:-:-:-:-:-:-:-:-:-:-:-:#
-
-#This module requires TIDoS Framework
-#https://github.com/theInfectedDrake/TIDoS-Framework
-
-pass
diff --git a/core/Vulnlysis/Misc_Bugs/webbugs.py b/core/Vulnlysis/Misc_Bugs/webbugs.py
deleted file mode 100644
index 6486f257..00000000
--- a/core/Vulnlysis/Misc_Bugs/webbugs.py
+++ /dev/null
@@ -1,202 +0,0 @@
-#!/usr/bin/env python2
-# coding: utf-8
-
-#-:-:-:-:-:-:-:-:-:-:-:-:#
-# TIDoS Framework #
-#-:-:-:-:-:-:-:-:-:-:-:-:#
-
-#This module requires TIDoS Framework
-#https://github.com/theInfectedDrake/TIDoS-Framework
-
-from __future__ import print_function
-import time
-import os
-import sys
-sys.path.append('modules/0x03-Vulnerability+Analysis/0x02-MisconfigurationBugs/')
-
-from icors import *
-from ssscript import *
-from clickjack import *
-from zone import *
-from webbugsban import *
-from hhi import *
-from netmisc import *
-from cloudflaremisc import *
-from hsts import *
-from sessionfix import *
-from headers import *
-from colors import *
-from xsstrace import *
-from cookiecheck import *
-from mailspoof import *
-
-def webbugs(web):
-
- print(W+'\n [*] Type Selected : Basic Web Bugs and Misconfigurations...')
- webbugsban()
- v = raw_input(O+B+' [#] \033[4mTID\033[1;0m '+GR+':> ' + color.END)
- print('\n')
- if v.strip() == '1':
- print(B+' [!] Type Selected :'+C+' iCORS')
- icors(web)
- raw_input(O+' [#] Press '+GR+'Enter'+O+' to continue...')
- webbugs(web)
-
- elif v.strip() == '2':
- print(B+' [!] Type Selected :'+C+' Same Site Scripting')
- ssscript(web)
- raw_input(O+' [#] Press '+GR+'Enter'+O+' to continue...')
- webbugs(web)
-
- elif v.strip() == '3':
- print(B+' [!] Type Selected :'+C+' Clickjack')
- clickjack(web)
- raw_input(O+' [#] Press '+GR+'Enter'+O+' to continue...')
- webbugs(web)
-
- elif v.strip() == '4':
- print(B+' [!] Type Selected :'+C+' Zone Transfer')
- zone(web)
- raw_input(O+' [#] Press '+GR+'Enter'+O+' to continue...')
- webbugs(web)
-
- elif v.strip() == '5':
- print(B+' [!] Type Selected :'+C+' Cookie Check')
- cookiecheck(web)
- raw_input(O+' [#] Press '+GR+'Enter'+O+' to continue...')
- webbugs(web)
-
- elif v.strip() == '6':
- print(B+' [!] Type Selected :'+C+' Sec. Headers')
- headers(web)
- raw_input(O+' [#] Press '+GR+'Enter'+O+' to continue...')
- webbugs(web)
-
- elif v.strip() == '7':
- print(B+' [!] Type Selected :'+C+' Cloudflare Misconfig.')
- cloudflaremisc(web)
- raw_input(O+' [#] Press '+GR+'Enter'+O+' to continue...')
- webbugs(web)
-
- elif v.strip() == '8':
- print(B+' [!] Type Selected :'+C+' HSTS Check')
- hsts(web)
- raw_input(O+' [#] Press '+GR+'Enter'+O+' to continue...')
- webbugs(web)
-
- elif v.strip() == '9':
- print(B+' [!] Type Selected :'+C+' Cross Site Tracing')
- xsstrace(web)
- raw_input(O+' [#] Press '+GR+'Enter'+O+' to continue...')
- webbugs(web)
-
- elif v.strip() == '10':
- print(B+' [!] Type Selected :'+C+' Telnet Enabled')
- netmisc(web)
- raw_input(O+' [#] Press '+GR+'Enter'+O+' to continue...')
- webbugs(web)
-
- elif v.strip() == '11':
- print(B+' [!] Type Selected :'+C+' Email Spoof')
- mailspoof(web)
- raw_input(O+' [#] Press '+GR+'Enter'+O+' to continue...')
- webbugs(web)
-
- elif v.strip() == '12':
- print(B+' [!] Type Selected :'+C+' Host Header Injection')
- hhi(web)
- raw_input(O+' [#] Press '+GR+'Enter'+O+' to continue...')
- webbugs(web)
-
- elif v.strip() == '13':
- print(B+' [!] Type Selected :'+C+' Cookie Injection')
- sessionfix(web)
- raw_input(O+' [#] Press '+GR+'Enter'+O+' to continue...')
- webbugs(web)
-
- elif v.strip() == 'A':
- print(B+' [!] Type Selected : All Modules')
- time.sleep(0.5)
-
- print(B+' [*] Firing up module -->'+C+' iCORS')
- icors(web)
- print(B+'\n [!] Module Completed -->'+C+' iCORS\n')
- time.sleep(1)
-
- print(B+' [*] Firing up module -->'+C+' SSS ')
- ssscript(web)
- print(B+'\n [!] Module Completed -->'+C+' SSS \n')
- time.sleep(1)
-
- print(B+' [*] Firing up module -->'+C+' ClickJacking')
- clickjack(web)
- print(B+'\n [!] Module Completed -->'+C+' ClickJacking\n')
- time.sleep(1)
-
- print(B+' [*] Firing up module -->'+C+' Zone Transfer')
- zone(web)
- print(B+'\n [!] Module Completed -->'+C+' Zone Transfer\n')
- time.sleep(1)
-
- print(B+' [*] Firing up module -->'+C+' Cookie Check')
- cookiecheck(web)
- print(B+'\n [!] Module Completed -->'+C+' Cookie Check\n')
- time.sleep(1)
-
- print(B+' [*] Firing up module -->'+C+' Security Headers ')
- headers(web)
- print(B+'\n [!] Module Completed -->'+C+' Security Headers \n')
- time.sleep(1)
-
- print(B+' [*] Firing up module -->'+C+' Cloudflare Misconfig.')
- cloudflaremisc(web)
- print(B+'\n [!] Module Completed -->'+C+' Cloudflare Misconfig.\n')
- time.sleep(1)
-
- print(B+' [*] Firing up module -->'+C+' Mail Spoofing')
- mailspoof(web)
- print(B+'\n [!] Module Completed -->'+C+' Mail Spoofing\n')
- time.sleep(0.5)
-
- print(B+' [*] Firing up module -->'+C+' HSTS Checker')
- hsts(web)
- print(B+'\n [!] Module Completed -->'+C+' HSTS Checker\n')
- time.sleep(0.5)
-
- print(B+' [*] Firing up module -->'+C+' Cross Site Tracing')
- xsstrace(web)
- print(B+'\n [!] Module Completed -->'+C+' Cross Site Tracing\n')
- time.sleep(0.5)
-
- print(B+' [*] Firing up module -->'+C+' Telnet Enabled')
- netmisc(web)
- print(B+'\n [!] Module Completed -->'+C+' Telnet Enabled\n')
- time.sleep(1)
-
- print(B+' [*] Firing up module -->'+C+' Host Header Injection')
- hhi(web)
- print(B+'\n [!] Module Completed -->'+C+' Host Header Injection\n')
- time.sleep(1)
-
- print(B+' [*] Firing up module -->'+C+' Cookie Injection')
- sessionfix(web)
- print(B+'\n [!] Module Completed -->'+C+' Cookie Injection\n')
- time.sleep(1)
-
- print(G+' [+] All modules successfully completed!')
- time.sleep(0.5)
- raw_input(O+' [#] Press '+GR+'Enter'+O+' to continue...')
- webbugs(web)
-
- elif v.strip() == '99':
- print('[!] Back')
- time.sleep(0.7)
-
- else:
- print('')
- dope = ['You high dude?','Hey there! Enter a valid option','Whoops! Thats not an option','Sorry fam! You just typed shit']
- print(dope[randint(0,3)])
- time.sleep(0.7)
- os.system('clear')
- webbugs(web)
-
diff --git a/core/Vulnlysis/Misc_Bugs/webbugsban.py b/core/Vulnlysis/Misc_Bugs/webbugsban.py
deleted file mode 100644
index 968beced..00000000
--- a/core/Vulnlysis/Misc_Bugs/webbugsban.py
+++ /dev/null
@@ -1,50 +0,0 @@
-#!/usr/bin/env python2
-# -*- coding: utf-8 -*-
-
-#-:-:-:-:-:-:-:-:-:-:-:-:#
-# TIDoS Framework #
-#-:-:-:-:-:-:-:-:-:-:-:-:#
-
-#This module requires TIDoS Framework
-#https://github.com/theInfectedDrake/TIDoS-Framework
-
-from __future__ import print_function
-import time
-import os
-from colors import *
-from core.Core.arts import bugsban_art
-
-def webbugsban():
-
- os.system('clear')
- print(bugsban_art)
- print(O+' [1]'+GR+' Insecure Cross Origin Resource Sharing'+W+' (Absolute)')
- time.sleep(0.1)
- print(O+' [2]'+GR+' Same Site Scripting'+W+' (Sub-Domains Based)')
- time.sleep(0.1)
- print(O+' [3]'+GR+' Clickjackable Vulnerabilities '+W+'(Framable Response)')
- time.sleep(0.1)
- print(O+' [4]'+GR+' Zone Transfer Vulnerabilities'+W+' (DNS Based)')
- time.sleep(0.1)
- print(O+' [5]'+GR+' Security on Cookies '+W+'(HTTPOnly & Secure Flags)')
- time.sleep(0.1)
- print(O+' [6]'+GR+' Security Headers Analysis '+W+'(Absolute)')
- time.sleep(0.1)
- print(O+' [7]'+GR+' Cloudflare Misconfiguration '+W+'(Get Real IP) ')
- time.sleep(0.1)
- print(O+' [8]'+GR+' HTTP Strict Transport Security Usage ')
- time.sleep(0.1)
- print(O+' [9]'+GR+' Cross-Site Tracing '+W+'(Port Based)')
- time.sleep(0.1)
- print(O+' [10]'+GR+' Network Security Misconfig. '+W+'(Telnet Port Based)')
- time.sleep(0.1)
- print(O+' [11]'+GR+' Spoofable Emails '+W+'(Missing SPF & DMARC Records) ')
- time.sleep(0.1)
- print(O+' [12]'+GR+' Host Header Injection '+W+'(Port Based) ')
- time.sleep(0.1)
- print(O+' [13]'+GR+' Cookie Injection '+W+'(Session Fixation)\n ')
- time.sleep(0.1)
- print(O+' [A]'+GR+' Load all the modules 1 by 1\n')
- time.sleep(0.1)
- print(O+' [99]'+GR+' Back\n')
-
diff --git a/core/Vulnlysis/Oth_Bugs/__init__.py b/core/Vulnlysis/Oth_Bugs/__init__.py
deleted file mode 100644
index 73a0d197..00000000
--- a/core/Vulnlysis/Oth_Bugs/__init__.py
+++ /dev/null
@@ -1,11 +0,0 @@
-#!/usr/bin/env python2
-# coding: utf-8
-
-#-:-:-:-:-:-:-:-:-:-:-:-:#
-# TIDoS Framework #
-#-:-:-:-:-:-:-:-:-:-:-:-:#
-
-#This module requires TIDoS Framework
-#https://github.com/theInfectedDrake/TIDoS-Framework
-
-pass
diff --git a/core/Vulnlysis/Oth_Bugs/othbugs.py b/core/Vulnlysis/Oth_Bugs/othbugs.py
deleted file mode 100644
index f3591fe1..00000000
--- a/core/Vulnlysis/Oth_Bugs/othbugs.py
+++ /dev/null
@@ -1,142 +0,0 @@
-#!/usr/bin/env python2
-# -*- coding: utf-8 -*-
-
-#-:-:-:-:-:-:-:-:-:-:-:-:#
-# TIDoS Framework #
-#-:-:-:-:-:-:-:-:-:-:-:-:#
-
-#Author : @_tID
-#This module requires TIDoS Framework
-#https://github.com/theInfectedDrake/TIDoS-Framework
-
-from __future__ import print_function
-import sys
-import os
-import time
-import subprocess
-import random
-from random import randint
-sys.path.append('modules/0x03-Vulnerability+Analysis/0x03-OtherWebBugs/')
-from popbrute import *
-from ftpbrute import *
-from sqlbrute import *
-from sshbrute import *
-from othbugsban import *
-from colors import *
-from smtpbrute import *
-from xmppbrute import *
-from telnetbrute import *
-
-def othbugs(web):
-
- print(" [!] Module Selected : Bruteforce Modules\n\n")
- othbugsban()
- print('')
- time.sleep(0.3)
- v = raw_input (GR+' [#] \033[1;4mTID\033[0m'+GR+' :> ' + color.END)
- print('')
- if v == '1':
- print(B+' [!] Type Selected :'+C+' FTP Brute')
- ftpbrute(web)
- raw_input(O+' [#] Press '+GR+'Enter'+O+' to continue...')
- print('\n\n')
- othbugs(web)
-
- elif v == '2':
- print(B+' [!] Type Selected :'+C+' SSH Brute')
- sshbrute(web)
- raw_input(O+' [#] Press '+GR+'Enter'+O+' to continue...')
- print('\n\n')
- othbugs(web)
-
- elif v == '3':
- print(B+' [!] Type Selected :'+C+' SQL Brute')
- sqlbrute(web)
- raw_input(O+' [#] Press '+GR+'Enter'+O+' to continue...')
- print('\n\n')
- othbugs(web)
-
- elif v == '4':
- print(B+' [!] Type Selected :'+C+' POP 3/2 Brute')
- popbrute(web)
- raw_input(O+' [#] Press '+GR+'Enter'+O+' to continue...')
- print('\n\n')
- othbugs(web)
-
- elif v == '5':
- print(B+' [!] Type Selected :'+C+' SMTP Brute')
- smtpbrute(web)
- raw_input(O+' [#] Press '+GR+'Enter'+O+' to continue...')
- print('\n\n')
- othbugs(web)
-
- elif v == '6':
- print(B+' [!] Type Selected :'+C+' TELNET Brute')
- telnetbrute(web)
- raw_input(O+' [#] Press '+GR+'Enter'+O+' to continue...')
- print('\n\n')
- othbugs(web)
-
- elif v == '7':
- print(B+' [!] Type Selected :'+C+' XMPP Brute')
- xmppbrute(web)
- raw_input(O+' [#] Press '+GR+'Enter'+O+' to continue...')
- print('\n\n')
- othbugs(web)
-
- elif v == 'A':
- print(B+' [!] Type Selected :'+C+' All Modules')
- time.sleep(0.5)
- print(B+' [*] Firing up module -->'+C+' FTP Brute')
- ftpbrute(web)
- print(B+' [!] Module Completed -->'+C+' FTP Brute\n')
- time.sleep(2)
-
- print(B+' [*] Firing up module -->'+C+' SSH Brute')
- sshbrute(web)
- print(B+' [!] Module Completed -->'+C+' SSH Brute\n')
- time.sleep(2)
-
- print(B+' [*] Firing up module -->'+C+' SQL Brute')
- sqlbrute(web)
- print(B+' [!] Module Completed -->'+C+' SQL Brute\n')
- time.sleep(2)
-
- print(B+' [*] Firing up module -->'+C+' POP Brute')
- popbrute(web)
- print(B+' [!] Module Completed -->'+C+' POP Brute\n')
- time.sleep(2)
-
- print(B+' [*] Firing up module -->'+C+' SMTP Brute')
- smtpbrute(web)
- print(B+' [!] Module Completed -->'+C+' SMTP Brute\n')
- time.sleep(2)
-
- print(B+' [*] Firing up module -->'+C+' TElNET Brute')
- telnetbrute(web)
- print(B+' [!] Module Completed -->'+C+' TELNET Brute\n')
- time.sleep(2)
-
- print(B+' [*] Firing up module -->'+C+' XMPP Brute')
- xmppbrute(web)
- print(B+' [!] Module Completed -->'+C+' XMPP Brute\n')
- time.sleep(2)
-
- print(B+' [!] All scantypes have been tested on target...')
- time.sleep(4)
- raw_input(O+' [#] Press '+GR+'Enter'+O+' to continue...')
- print(B+' [*] Going back to menu...')
- othbugs(web)
-
- elif v == '99':
- print(GR+' [*] Going back...')
- time.sleep(0.5)
- os.system('clear')
-
- else:
- dope = ['You high dude?','Shit! Enter a valid option','Whoops! Thats not an option','Sorry! You just typed shit']
- print(dope[randint(0,3)])
- time.sleep(0.7)
- os.system('clear')
- othbugs(web)
-
diff --git a/core/Vulnlysis/Oth_Bugs/othbugsban.py b/core/Vulnlysis/Oth_Bugs/othbugsban.py
deleted file mode 100644
index 801e3d30..00000000
--- a/core/Vulnlysis/Oth_Bugs/othbugsban.py
+++ /dev/null
@@ -1,40 +0,0 @@
-#!/usr/bin/env python2
-# coding: utf-8
-
-#-:-:-:-:-:-:-:-:-:-:-:-:#
-# TIDoS Framework #
-#-:-:-:-:-:-:-:-:-:-:-:-:#
-
-#Author : @_tID
-#This module requires TIDoS Framework
-#https://github.com/theInfectedDrake/TIDoS-Framework
-
-from __future__ import print_function
-from core.Core.arts import bugsban_art
-from colors import *
-import time
-
-def othbugsban():
-
- time.sleep(0.4)
- os.system('clear')
- print(bugsban_art)
- time.sleep(0.3)
- print(B+'\n [1]'+C+' FTP Protocol Bruteforce'+W+' (Default Credentials)')
- time.sleep(0.1)
- print(B+' [2]'+C+' SSH Protocol Bruteforce'+W+' (Default Credentials)')
- time.sleep(0.1)
- print(B+' [3]'+C+' SQL Protocol Bruteforce'+W+' (Default Credentials)')
- time.sleep(0.1)
- print(B+' [4]'+C+' POP 3/2 Protocol Bruteforce'+W+' (Default Credentials)')
- time.sleep(0.1)
- print(B+' [5]'+C+' SMTP Protocol Bruteforce'+W+' (Default Credentials)')
- time.sleep(0.1)
- print(B+' [6]'+C+' TELNET Protocol Bruteforce'+W+' (Default Credentials)')
- time.sleep(0.1)
- print(B+' [7]'+C+' XMPP Protocol Bruteforce'+W+' (Default Credentials)\n')
- time.sleep(0.1)
- print(B+' [A]'+C+' Start up ALL modules 1 by 1\n')
- time.sleep(0.1)
- print(B+' [99]'+C+' Back\n')
-
diff --git a/core/Vulnlysis/Serio_Bugs/__init__.py b/core/Vulnlysis/Serio_Bugs/__init__.py
deleted file mode 100644
index 73a0d197..00000000
--- a/core/Vulnlysis/Serio_Bugs/__init__.py
+++ /dev/null
@@ -1,11 +0,0 @@
-#!/usr/bin/env python2
-# coding: utf-8
-
-#-:-:-:-:-:-:-:-:-:-:-:-:#
-# TIDoS Framework #
-#-:-:-:-:-:-:-:-:-:-:-:-:#
-
-#This module requires TIDoS Framework
-#https://github.com/theInfectedDrake/TIDoS-Framework
-
-pass
diff --git a/core/Vulnlysis/Serio_Bugs/serbugs.py b/core/Vulnlysis/Serio_Bugs/serbugs.py
deleted file mode 100644
index 0d7cf7ab..00000000
--- a/core/Vulnlysis/Serio_Bugs/serbugs.py
+++ /dev/null
@@ -1,225 +0,0 @@
-#!/usr/bin/env python2
-# -*- coding: utf-8 -*-
-
-#-:-:-:-:-:-:-:-:-:-:-:-:#
-# TIDoS Framework #
-#-:-:-:-:-:-:-:-:-:-:-:-:#
-
-#Author: @_tID
-#This module requires TIDoS Framework
-#https://github.com/theInfectedDrake/TIDoS-Framework
-
-from __future__ import print_function
-import sys
-import time
-import os
-sys.path.append('modules/0x03-Vulnerability+Analysis/0x01-TopPriorityBugs/')
-
-from serbugsban import *
-from lfi import *
-from rfi import *
-from ldap import *
-from rce import *
-from csrf import *
-from sqli import *
-from crlf import *
-from subdomover import *
-from strutsshock import *
-from phpi import *
-from htmli import *
-from xpathi import *
-from shellshock import *
-from colors import *
-from xss import *
-from openredirect import *
-from pathtrav import *
-
-def serbugs(web):
-
- print(GR+'\n [*] Loading module...')
- serbugsban()
- v = raw_input(''+O+' \033[4mTID\033[1;0m '+GR+':> ' + color.END)
- print('\n')
- if v.strip() == '1':
- print(' Type Selected : LFI')
- lfi(web)
- raw_input(O+' [#] Press '+GR+'Enter'+O+' to continue...')
- serbugs(web)
-
- elif v.strip() == '2':
- print(' Type Selected : RFI')
- rfi(web)
- raw_input(O+' [#] Press '+GR+'Enter'+O+' to continue...')
- serbugs(web)
-
- elif v.strip() == '3':
- print(' Type Selected : RCE')
- rce(web)
- raw_input(O+' [#] Press '+GR+'Enter'+O+' to continue...')
- serbugs(web)
-
- elif v.strip() == '4':
- print(' Type Selected : Path Traversal')
- pathtrav(web)
- raw_input(O+' [#] Press '+GR+'Enter'+O+' to continue...')
- serbugs(web)
-
- elif v.strip() == '5':
- print(' Type Selected : CSRF')
- csrf(web)
- raw_input(O+' [#] Press '+GR+'Enter'+O+' to continue...')
- serbugs(web)
-
- elif v.strip() == '6':
- print(' Type Selected : XSS')
- xss(web)
- raw_input(O+' [#] Press '+GR+'Enter'+O+' to continue...')
- serbugs(web)
-
- elif v.strip() == '7':
- print(' Type Selected : SQLi')
- sqli(web)
- raw_input(O+' [#] Press '+GR+'Enter'+O+' to continue...')
- serbugs(web)
-
- elif v.strip() == '8':
- print(' Type Selected : LDAP Injection')
- ldap(web)
- raw_input(O+' [#] Press '+GR+'Enter'+O+' to continue...')
- serbugs(web)
-
- elif v.strip() == '9':
- print(' Type Selected : HTML Code Injection')
- htmli(web)
- raw_input(O+' [#] Press '+GR+'Enter'+O+' to continue...')
- serbugs(web)
-
- elif v.strip() == '10':
- print(' Type Selected : HTTP Response Splitting')
- crlf(web)
- raw_input(O+' [#] Press '+GR+'Enter'+O+' to continue...')
- serbugs(web)
-
- elif v.strip() == '11':
- print(' Type Selected : PHP Code Injection')
- phpi(web)
- raw_input(O+' [#] Press '+GR+'Enter'+O+' to continue...')
- serbugs(web)
-
- elif v.strip() == '12':
- print(' Type Selected : XPATH Injection')
- xpathi(web)
- raw_input(O+' [#] Press '+GR+'Enter'+O+' to continue...')
- serbugs(web)
-
- elif v.strip() == '13':
- print(' Type Selected : Shellshock')
- shellshock(web)
- raw_input(O+' [#] Press '+GR+'Enter'+O+' to continue...')
- serbugs(web)
-
- elif v.strip() == '14':
- print(' Type Selected : Apache Struts Shock')
- strutsshock(web)
- raw_input(O+' [#] Press '+GR+'Enter'+O+' to continue...')
- serbugs(web)
-
- elif v.strip() == '15':
- print(' Type Selected : URL Validation')
- redirect(web)
- raw_input(O+' [#] Press '+GR+'Enter'+O+' to continue...')
- serbugs(web)
-
- elif v.strip() == '16':
- print(' Type Selected : Subdomain Takeover')
- subdomover(web)
- raw_input(O+' [#] Press '+GR+'Enter'+O+' to continue...')
- serbugs(web)
-
- elif v.strip() == 'A':
- print(' [!] Type Selected : All Modules')
- time.sleep(0.5)
-
- print(' [*] Firing up module --> LFI')
- lfi(web)
- print(' [!] Module Completed --> LFI\n')
- time.sleep(1)
-
- print(' [*] Firing up module --> RFI ')
- rfi(web)
- print(' [!] Module Completed --> RFI \n')
- time.sleep(1)
-
- print(' [*] Firing up module --> RCE')
- rce(web)
- print(' [!] Module Completed --> RCE\n')
- time.sleep(1)
-
- print(' [*] Firing up module --> Path Traversal')
- pathtrav(web)
- print(' [!] Module Completed --> Path Traversal\n')
- time.sleep(1)
-
- print(' [*] Firing up module --> CSRF')
- csrf(web)
- print(' [!] Module Completed --> CSRF\n')
- time.sleep(1)
-
- print(' [*] Firing up module --> XSS ')
- xss(web)
- print(' [!] Module Completed --> XSS \n')
- time.sleep(1)
-
- print(' [*] Firing up module --> SQLi')
- sqli(web)
- print(' [!] Module Completed --> SQLi\n')
- time.sleep(1)
-
- print(' [*] Firing up module --> LDAPi')
- ldap(web)
- print(' [!] Module Completed --> LDAPi\n')
- time.sleep(1)
-
- print(' [*] Firing up module --> CRLF')
- crlf(web)
- print(' [!] Module Completed --> CRLF\n')
- time.sleep(1)
-
- print(' [*] Firing up module --> PHP Code Injection')
- phpi(web)
- print(' [!] Module Completed --> PHP Code Injection\n')
- time.sleep(0.5)
-
- print(' [*] Firing up module --> XPATH Injection')
- xpathi(web)
- print(' [!] Module Completed --> XPATH Injection\n')
- time.sleep(0.5)
-
- print(' [*] Firing up module --> ShellShock')
- shellshock(web)
- print(' [!] Module Completed --> ShellShock\n')
- time.sleep(1)
-
- print(' [*] Firing up module --> URL Forwards')
- redirect(web)
- print(' [!] Module Completed --> Url Forwards\n')
- time.sleep(0.5)
- print(G+' [+] All modules successfully completed!')
- time.sleep(4)
- raw_input(O+' [#] Press '+GR+'Enter'+O+' to continue...')
- serbugs(web)
-
- elif v.strip() == '99':
- print('[!] Back')
- time.sleep(0.7)
-
- else:
- print('')
- dope = ['You high dude?','Hey there! Enter a valid option','Whoops! Thats not an option','Sorry fam! You just typed shit']
- print(dope[randint(0,3)])
- time.sleep(0.7)
- os.system('clear')
-
- time.sleep(1)
- serbugs(web)
-
diff --git a/core/Vulnlysis/Serio_Bugs/serbugsban.py b/core/Vulnlysis/Serio_Bugs/serbugsban.py
deleted file mode 100644
index 55a0e3c9..00000000
--- a/core/Vulnlysis/Serio_Bugs/serbugsban.py
+++ /dev/null
@@ -1,57 +0,0 @@
-#!/usr/bin/env python2
-# -*- coding: utf-8 -*-
-
-#-:-:-:-:-:-:-:-:-:-:-:-:#
-# TIDoS Framework #
-#-:-:-:-:-:-:-:-:-:-:-:-:#
-
-#This module requires TIDoS Framework
-#https://github.com/theInfectedDrake/TIDoS-Framework
-
-from __future__ import print_function
-import time
-from colors import *
-import os
-from core.Core.arts import bugsban_art
-
-def serbugsban():
-
- time.sleep(0.5)
- os.system('clear')
- print(bugsban_art)
- print(B+' [1]'+C+' Local File Inclusion'+W+' (Root Directories)')
- time.sleep(0.1)
- print(B+' [2]'+C+' Remote File Inclusion'+W+' (Executable Scripts)')
- time.sleep(0.1)
- print(B+' [3]'+C+' OS Command Injection'+W+' (Windows & Linux)')
- time.sleep(0.1)
- print(B+' [4]'+C+' Path Traversal '+W+'(Sensitive Paths)')
- time.sleep(0.1)
- print(B+' [5]'+C+' Cross-Site Request Forgery '+W+'(Absolute)')
- time.sleep(0.1)
- print(B+' [6]'+C+' Cross-Site Scripting '+W+'(Absolute)')
- time.sleep(0.1)
- print(B+' [7]'+C+' SQL Injection '+W+'(Error & Blind Based)')
- time.sleep(0.1)
- print(B+' [8]'+C+' LDAP Entity Injection '+W+'(Error Enumeration)')
- time.sleep(0.1)
- print(B+' [9]'+C+' HTML Code Injection '+W+'(Tag Based)')
- time.sleep(0.1)
- print(B+' [10]'+C+' HTTP Response Splitting '+W+'(CRLF Injection)')
- time.sleep(0.1)
- print(B+' [11]'+C+' PHP Code Injection '+W+'(Windows + Linux)')
- time.sleep(0.1)
- print(B+' [12]'+C+' XPATH Injection '+W+'(Blind and Error Based)')
- time.sleep(0.1)
- print(B+' [13]'+C+' Shellshock Vulnerabilities'+W+' (Bash RCE)')
- time.sleep(0.1)
- print(B+' [14]'+C+' Apache Struts Shock'+W+' (Apache RCE)')
- time.sleep(0.1)
- print(B+' [15]'+C+' Unvalidated URL Redirects'+W+' (Open Redirects)')
- time.sleep(0.1)
- print(B+' [16]'+C+' Sub-domain Takeover'+W+' (50+ Services)\n')
- time.sleep(0.1)
- print(B+' [A]'+C+' Load all the modules 1 by 1\n')
- time.sleep(0.1)
- print(B+' [99]'+C+' Back\n')
-
diff --git a/core/Vulnlysis/__init__.py b/core/Vulnlysis/__init__.py
deleted file mode 100644
index 73a0d197..00000000
--- a/core/Vulnlysis/__init__.py
+++ /dev/null
@@ -1,11 +0,0 @@
-#!/usr/bin/env python2
-# coding: utf-8
-
-#-:-:-:-:-:-:-:-:-:-:-:-:#
-# TIDoS Framework #
-#-:-:-:-:-:-:-:-:-:-:-:-:#
-
-#This module requires TIDoS Framework
-#https://github.com/theInfectedDrake/TIDoS-Framework
-
-pass
diff --git a/core/Vulnlysis/colors.py b/core/Vulnlysis/colors.py
deleted file mode 100644
index 96ed4433..00000000
--- a/core/Vulnlysis/colors.py
+++ /dev/null
@@ -1,37 +0,0 @@
-#!/usr/bin/env python2
-
-#-:-:-:-:-:-:-:-:-:-:-:-:#
-# TIDoS Framework #
-#-:-:-:-:-:-:-:-:-:-:-:-:#
-
-#This module requires TIDoS Framework
-#https://github.com/theInfectedDrake/TIDoS-Framework
-
-###############################
-class color:
- PURPLE = '\033[95m'
- CYAN = '\033[96m'
- DARKCYAN = '\033[36m'
- BLUE = '\033[94m'
- GREEN = '\033[92m'
- YELLOW = '\033[93m'
- RED = '\033[91m'
- BOLD = '\033[1m'
- UNDERLINE = '\033[4m'
- END = '\033[0m'
- HEADER = '\033[95m'
- OKBLUE = '\033[94m'
- OKGREEN = '\033[92m'
- WARNING = '\033[93m'
- FAIL = '\033[91m'
-
-W = '\033[1;0m' # white (normal)
-R = '\033[1;31m' # red
-G = '\033[1;32m' # green
-O = '\033[1;33m' # orange
-B = '\033[1;34m' # blue
-P = '\033[1;35m' # purple
-C = '\033[1;36m' # cyan
-GR = '\033[1;37m' # gray
-T = '\033[1;93m' # tan
-###############################
diff --git a/core/Vulnlysis/vuln.py b/core/Vulnlysis/vuln.py
deleted file mode 100644
index c7ec1210..00000000
--- a/core/Vulnlysis/vuln.py
+++ /dev/null
@@ -1,49 +0,0 @@
-#!/usr/bin/env python2
-
-#-:-:-:-:-:-:-:-:-:-:-:-:#
-# TIDoS Framework #
-#-:-:-:-:-:-:-:-:-:-:-:-:#
-
-#This module requires TIDoS Framework
-#https://github.com/theInfectedDrake/TIDoS-Framework
-
-from __future__ import print_function
-import os
-import time
-import random
-from random import *
-from vulnban import *
-from core.Vulnlysis.Misc_Bugs.webbugs import *
-from core.Vulnlysis.Oth_Bugs.othbugs import *
-from core.Vulnlysis.Serio_Bugs.serbugs import *
-from colors import *
-
-def vuln(web):
-
- print(B+' [+] Module Loaded : '+C+'Vulnerability Analysis')
- vulnban()
- v = raw_input(''+O+' \033[4mTID\033[1;0m '+GR+':> ' + color.END)
- print('\n')
-
- if v.strip() == '1':
- webbugs(web)
-
- elif v.strip() == '2':
- serbugs(web)
-
- elif v.strip() == '3':
- othbugs(web)
-
- elif v.strip() == '99':
- print('[!] Back')
- time.sleep(0.7)
- os.system('clear')
-
- else:
- dope = ['You high dude?','Hey there! Enter a valid option','Whoops! Thats not an option','Sorry fam! You just typed shit']
- print(dope[randint(0,3)])
- time.sleep(0.7)
- os.system('clear')
- vulnban()
- vuln(web)
-
diff --git a/core/Vulnlysis/vulnban.py b/core/Vulnlysis/vulnban.py
deleted file mode 100644
index c894dde1..00000000
--- a/core/Vulnlysis/vulnban.py
+++ /dev/null
@@ -1,30 +0,0 @@
-#!/usr/bin/env python2
-# coding: utf-8
-
-#-:-:-:-:-:-:-:-:-:-:-:-:#
-# TIDoS Framework #
-#-:-:-:-:-:-:-:-:-:-:-:-:#
-
-#This module requires TIDoS Framework
-#https://github.com/theInfectedDrake/TIDoS-Framework
-
-from __future__ import print_function
-import os, time
-from colors import *
-from core.Core.arts import vulnban_art
-
-def vulnban():
-
- time.sleep(0.3)
- os.system('clear')
- print(O+' [!] Module Selected : Vulnerability Analysis')
- print(vulnban_art)
- print(B+' [1] '+C+' Basic Bugs & Misconfigurations'+W+' (Low Priority [P0x3-P0x4])')
- time.sleep(0.2)
- print(B+' [2] '+C+' Critical Vulnerabilities '+W+'(High Priority [P0x1-P0x2])')
- time.sleep(0.2)
- print(B+' [3] '+C+' Others '+W+'(Bruters)\n')
- time.sleep(0.2)
- print(B+' [99] '+C+'Back\n')
- time.sleep(0.2)
-
diff --git a/core/__init__.py b/core/__init__.py
index 73a0d197..8d1c8b69 100644
--- a/core/__init__.py
+++ b/core/__init__.py
@@ -1,11 +1 @@
-#!/usr/bin/env python2
-# coding: utf-8
-
-#-:-:-:-:-:-:-:-:-:-:-:-:#
-# TIDoS Framework #
-#-:-:-:-:-:-:-:-:-:-:-:-:#
-
-#This module requires TIDoS Framework
-#https://github.com/theInfectedDrake/TIDoS-Framework
-
-pass
+
diff --git a/core/__pycache__/__init__.cpython-37.pyc b/core/__pycache__/__init__.cpython-37.pyc
new file mode 100644
index 00000000..33b2aef7
Binary files /dev/null and b/core/__pycache__/__init__.cpython-37.pyc differ
diff --git a/core/__pycache__/colors.cpython-37.pyc b/core/__pycache__/colors.cpython-37.pyc
new file mode 100644
index 00000000..697a5602
Binary files /dev/null and b/core/__pycache__/colors.cpython-37.pyc differ
diff --git a/core/__pycache__/variables.cpython-37.pyc b/core/__pycache__/variables.cpython-37.pyc
new file mode 100644
index 00000000..cf485848
Binary files /dev/null and b/core/__pycache__/variables.cpython-37.pyc differ
diff --git a/core/database/database_module.py b/core/database/database_module.py
new file mode 100644
index 00000000..1a7c73ea
--- /dev/null
+++ b/core/database/database_module.py
@@ -0,0 +1,113 @@
+import sqlite3
+
+# This module contains three functions:
+# 1. save_data - inserts data into the database, and automatically creates tables and
+# organizes the data as necessary.
+# 2. retrieve_data - generates SQLite queries to retrieve the requested data from the
+# database and returns the data.
+# 3. get_info - reads the database and returns a dictionary that acts as an index to
+# the data in the database. This dictionary is used by the menu function to build
+# out the menu.
+# The functions in this module take the following arguments:
+# save_data and retrieve_data take the following arguments:
+# database = database file name in the format "databasename.db". This will be the same
+# throughout the session.
+# module = the name of the top-level module you're in. Can't have spaces or special
+# characters. Should be EXACTLY matching one of the following:
+# "ReconANDOSINT", "ScanANDEnum", "VulnAnalysis", "Exploitation", or "AuxModules"
+# lvl1, lvl2, lvl3 = the names of the next levels of modules, on down to the one you're
+# in. If you don't go all the way down to lvl3, input Null. Can have spaces. Should
+# exactly match the name of the module as written in the menu.
+# data = should be the data generated by the module. It will be unchanged when it is
+# passed in and out of the database.
+
+def save_data(database, module, lvl1, lvl2, lvl3, host, data):
+ connection = sqlite3.connect(database)
+ cursor = connection.cursor()
+ check_table = 'SELECT name FROM sqlite_master WHERE type=\'table\' AND name=\'{}\''.format(module)
+ cursor.execute(check_table)
+ checker = cursor.fetchone()
+ if checker == None:
+ create_table = 'CREATE TABLE {} (\
+ id INTEGER PRIMARY KEY, \
+ lvl1 text not null, \
+ lvl2 text, \
+ lvl3 text, \
+ number int, \
+ data blob);'.format(module)
+ cursor.execute(create_table)
+ connection.commit()
+ check_entries = 'SELECT number FROM {} WHERE lvl1="{}" AND lvl2="{}" AND lvl3="{}"'.format(module, lvl1, lvl2, lvl3)
+ cursor.execute(check_entries)
+ printable = cursor.fetchall()
+ ctr = 0
+ high = []
+ while ctr < len(printable):
+ high.append(printable[ctr][0])
+ ctr += 1
+ if high == []:
+ highest = 1
+ else:
+ highest = max(high) + 1
+ insert_into_table = 'INSERT INTO {} (lvl1, lvl2, lvl3, number, data) VALUES (?, ?, ?, ?, ?);'.format(module)
+ table_entry_parameters = (lvl1, lvl2, lvl3, highest, data)
+ cursor.execute(insert_into_table, table_entry_parameters)
+ connection.commit()
+ connection.close()
+
+def retrieve_data(database, module, lvl1, lvl2, lvl3, num):
+ connection = sqlite3.connect(database)
+ cursor = connection.cursor()
+ if lvl3 == "":
+ if lvl2 == "":
+ select_from_table = 'SELECT data FROM {} WHERE lvl1="{}" AND lvl2="" AND lvl3="" AND number={}'.format(module, lvl1, num)
+ else:
+ select_from_table = 'SELECT data FROM {} WHERE lvl1="{}" AND lvl2="{}" AND lvl3="" AND number={}'.format(module, lvl1, lvl2, num)
+ else:
+ select_from_table = 'SELECT data FROM {} WHERE lvl1="{}" AND lvl2="{}" AND lvl3="{}" AND number={}'.format(module, lvl1, lvl2, lvl3, num)
+ cursor.execute(select_from_table)
+ returned_data = cursor.fetchall()
+ connection.close()
+ return returned_data[0][0]
+
+# This function receives only the file name of the database. It reads what is in the
+# database and returns a dictionary that acts as an index to the data saved in the
+# database. This dictionary is read by the menu function to build out the menu.
+def get_info(database):
+ modules = ['ReconANDOSINT', 'ScanANDEnum', 'VulnAnalysis', 'Exploitation', 'AuxModules']
+ saved_modules = []
+ data_saved = {}
+ connection = sqlite3.connect(database)
+ cursor = connection.cursor()
+ for module in modules:
+ check_table = 'SELECT name FROM sqlite_master WHERE type=\'table\' AND name=\'{}\''.format(module)
+ cursor.execute(check_table)
+ check = cursor.fetchone()
+ if check != None:
+ saved_modules.append(module)
+ for module in saved_modules:
+ if module not in data_saved.keys():
+ data_saved[module] = {}
+ ctr = 1
+ while True:
+ cmd = 'SELECT lvl1, lvl2, lvl3, number FROM {} WHERE id={}'.format(module, ctr)
+ cursor.execute(cmd)
+ grab_data = cursor.fetchone()
+ if grab_data == None:
+ break
+ else:
+ if grab_data[0] not in data_saved[module].keys():
+ data_saved[module][grab_data[0]] = {}
+ if grab_data[1] != '':
+ if grab_data[1] not in data_saved[module][grab_data[0]].keys():
+ data_saved[module][grab_data[0]][grab_data[1]] = {}
+ if grab_data[2] == '':
+ if grab_data[1] == '':
+ data_saved[module][grab_data[0]].update( {grab_data[3] : "Data"} )
+ else:
+ data_saved[module][grab_data[0]][grab_data[1]].update( {grab_data[3] : "Data"} )
+ else:
+ data_saved[module][grab_data[0]][grab_data[1]][grab_data[2]].update( {grab_data[3] : "Data"} )
+ ctr += 1
+ connection.close()
+ return data_saved
\ No newline at end of file
diff --git a/core/database/db_menu.py b/core/database/db_menu.py
new file mode 100644
index 00000000..1dc1ebaf
--- /dev/null
+++ b/core/database/db_menu.py
@@ -0,0 +1,108 @@
+from database_module import get_info, retrieve_data
+import sys
+
+menu_dict = {
+ 'ReconANDOSINT' : 'Reconnaissance & OSINT',
+ 'ScanANDEnum' : 'Scanning & Enumeration',
+ 'VulnAnalysis' : 'Vulnerability Analysis',
+ 'Exploitation' : 'Exploitation',
+ 'AuxModules' : 'Auxiliary Modules'
+ }
+db_name = "sessionresults.db"
+query_list = [db_name]
+error_message = "Sorry, please enter one of the choices listed."
+menu_message = "Please make your selection: "
+input_cursor = "Input > "
+main_exit_message = "E) Exit."
+higher_menu_exit_message = "0) Exit to higher menu."
+exit_message = "Now exiting. Goodbye."
+
+def query_db(query_list):
+ db, module, lvl1 = query_list[0], query_list[1], query_list[2]
+ if len(query_list) == 4:
+ lvl2, lvl3, num = "", "", query_list[3]
+ elif len(query_list) == 5:
+ lvl2, lvl3, num = query_list[3], "", query_list[4]
+ else:
+ lvl2, lvl3, num = query_list[3], query_list[4], query_list[5]
+ print(retrieve_data(db, module, lvl1, lvl2, lvl3, num))
+
+def build_db_menu(a_list, b_list, selection):
+ print(menu_message)
+ while True:
+ ctr = 0
+ while ctr < len(a_list):
+ if selection == "Data":
+ print("{}) Scan {}".format(str(ctr + 1), a_list[ctr]))
+ elif b_list != "None":
+ print("{}) {}".format(str(ctr + 1), b_list[ctr]))
+ else:
+ print("{}) {}".format(str(ctr + 1), a_list[ctr]))
+ ctr += 1
+ if selection != "None":
+ print(higher_menu_exit_message)
+ print(main_exit_message)
+ cmd = input(input_cursor)
+ if cmd == "e" or cmd == "E" or cmd == "exit" or cmd == "Exit" or cmd == "EXIT":
+ print(exit_message)
+ sys.exit()
+ elif cmd == "0":
+ del query_list[-1]
+ return cmd
+ elif int(cmd) <= len(a_list):
+ query_list.append(a_list[int(cmd) - 1])
+ if selection == "None":
+ return a_list[int(cmd) - 1]
+ elif selection[a_list[int(cmd) - 1]] == "Data":
+ print(selection[a_list[int(cmd) - 1]])
+ query_db(query_list)
+ del query_list[-1]
+ else:
+ return a_list[int(cmd) - 1]
+ else:
+ print(error_message)
+
+def level_five_menu(choice1, choice2, choice3, choice4, menu_data):
+ a_list = list(menu_data[choice1][choice2][choice3][choice4].keys())
+ selection = menu_data[choice1][choice2][choice3][choice4]
+ result = build_db_menu(a_list, "None", selection)
+ if result == "0":
+ level_four_menu(choice1, choice2, choice3, menu_data)
+
+def level_four_menu(choice1, choice2, choice3, menu_data):
+ a_list = list(menu_data[choice1][choice2][choice3].keys())
+ selection = menu_data[choice1][choice2][choice3]
+ result = build_db_menu(a_list, "None", selection)
+ if result == "0":
+ level_three_menu(choice1, choice2, menu_data)
+ else:
+ level_five_menu(choice1, choice2, choice3, result, menu_data)
+
+def level_three_menu(choice1, choice2, menu_data):
+ a_list = list(menu_data[choice1][choice2].keys())
+ selection = menu_data[choice1][choice2]
+ result = build_db_menu(a_list, "None", selection)
+ if result == "0":
+ level_two_menu(choice1, menu_data)
+ else:
+ level_four_menu(choice1, choice2, result, menu_data)
+
+def level_two_menu(choice, menu_data):
+ a_list = list(menu_data[choice].keys())
+ selection = menu_data[choice]
+ result = build_db_menu(a_list, "None", selection)
+ if result == "0":
+ menu(menu_data)
+ else:
+ level_three_menu(choice, result, menu_data)
+
+def menu(menu_data):
+ a_list = list(menu_data.keys())
+ b_list = []
+ for item in a_list:
+ b_list.append(menu_dict[item])
+ result = build_db_menu(a_list, b_list, "None")
+ level_two_menu(result, menu_data)
+
+menu_data = get_info(db_name)
+menu(menu_data)
\ No newline at end of file
diff --git a/core/database/tidos.db b/core/database/tidos.db
new file mode 100644
index 00000000..d4b04e9c
Binary files /dev/null and b/core/database/tidos.db differ
diff --git a/lib/bs4_mod/beautifulsoup4.egg-info/dependency_links.txt b/core/doc/.gitignore
similarity index 100%
rename from lib/bs4_mod/beautifulsoup4.egg-info/dependency_links.txt
rename to core/doc/.gitignore
diff --git a/core/doc/CONTRIBUTING b/core/doc/CONTRIBUTING
new file mode 100644
index 00000000..e679f8b5
--- /dev/null
+++ b/core/doc/CONTRIBUTING
@@ -0,0 +1,4 @@
+Contributing to TIDoS Framework
+---------------------------------
+
+
diff --git a/core/doc/DISCLAIMER b/core/doc/DISCLAIMER
new file mode 100644
index 00000000..17c1586e
--- /dev/null
+++ b/core/doc/DISCLAIMER
@@ -0,0 +1,4 @@
+DISCLAIMER
+----------
+
+TIDoS Attack was provided as an open-source, royalty-free penetration testing toolkit. It has capable modules in various phases which can unveil potential dangerous flaws in various web-applications which can further be exploited maliciously. Therefore the author as well as the contrbutors assume no liability for misuse of this toolkit. Usage of TIDoS Attack for testing or exploiting websites without prior mutual consent can be considered as an illegal activity. It is the final user's responsibility to obey all applicable local, state and federal laws.
diff --git a/agree b/core/doc/choice
similarity index 100%
rename from agree
rename to core/doc/choice
diff --git a/core/doc/local b/core/doc/local
new file mode 100644
index 00000000..8d1c8b69
--- /dev/null
+++ b/core/doc/local
@@ -0,0 +1 @@
+
diff --git a/core/doc/man/tidos.1 b/core/doc/man/tidos.1
new file mode 100644
index 00000000..211f0ce3
--- /dev/null
+++ b/core/doc/man/tidos.1
@@ -0,0 +1,144 @@
+.TH VAILE 1 2020-03-23 ""
+
+.SH NAME
+.P
+\fBVaile\fR — Metasploit\-like pentest framework derived from TIDoS
+
+.SH SYNOPSIS
+.P
+\fBVaile\fR [\fIOPTION\fR]...
+
+.SH DESCRIPTION
+.P
+\fBVaile\fR is a versatile collection of tools for \fIwebapp\fR security testing. It was designed in order to help penetration testers during all phases of an operation, especially reconnaissance, scanning and vulnerability analysis.
+
+.P
+It features 3 easy\-to\-use interfaces: an interactive shell, an inline CLI and a graphical user interface powered by Qt. Its modules are easily comprehensible and fully verbose, and are using \fImultiprocessing\fR to make attacking more efficient. If requested, TIDoS will also pipe attacks through \fITor\fR, rendering your attacks more anonymous and difficult to track.
+
+.P
+The \fBVaile\fR framework is module\-based, so users can easily create their own modules, or edit the built\-in ones, respecting the general structure of the already existent modules.
+
+.P
+Its structure allows it to evolve and be easily adapted to new features and modules. Currently, it has 108 modules ready to be used.
+
+.SH OPTIONS
+.P
+Running \fBVaile\fR without any argument runs an interactive, metasploit\-like shell, with all settings set to default.
+
+.TP
+You can also specify a target URL, combined with a module, to execute this module on given target in a quicker and easier way.
+\fB$ TIDoS \-v http://example.com \-l pathtrav\fR
+
+.TP
+You also can use a previously saved \fIsession\fR file as argument, to use it instead of a target URL. Sessions can (and need to) be saved with the \fBsessions\fR command.
+\fB$ TIDoS \-v saarsec \-l pathtrav \-s\fR
+
+.TP
+Using TIDoS's \fBVAL\fR file format, you can fully automate attacks, since it sets targets, modules and options automatically. Load a file up this way:
+\fB$ TIDoS \-c syn.val\fR
+
+.TP
+When attacking per CLI, you can also make use of the \fBtor\fR command to pipe attacks through the Tor network, like this:
+\fB$ TIDoS \-v http://example.com \-l pathtrav \-\-tor\fR
+
+.TP
+Finally, \fBVaile\fR can be run as a graphical user interface, using and wrapping around the core framework.
+\fB$ TIDoS \-\-app\fR
+
+.TP
+For inline help and more options, use \fBVaile \-\-help\fR
+
+.SH INTERFACE
+.P
+The main interface of \fBVaile\fR framework is its interactive, metasploit\-like console.
+
+.P
+\fBTO GET HELP\fR on a TIDoS command, you can use "\fBhelp 
+ 
+ 
+ 
+ 
+