scylla package

Subpackages

• scylla.providers package
  • Submodules
  • scylla.providers.a2u_provider module
  • scylla.providers.base_provider module
  • scylla.providers.cool_proxy_provider module
  • scylla.providers.data5u_provider module
  • scylla.providers.free_proxy_list_provider module
  • scylla.providers.http_proxy_provider module
  • scylla.providers.kuaidaili_provider module
  • scylla.providers.spys_me_provider module
  • scylla.providers.spys_one_provider module
  • scylla.providers.xici_provider module
  • Module contents
• scylla.web package
  • Submodules
  • scylla.web.server module
  • Module contents

Submodules

scylla.cli module

scylla.cli.app_main()

scylla.cli.handle_special_flags(args: dict)

scylla.cli.main(args) → int

scylla.config module

scylla.config.batch_set_config(**kwargs)

scylla.config.get_config(key: str, default: str = None) → Union[str, NoneType]

scylla.config.set_config(key: str, value: str)

scylla.database module

class scylla.database.BaseModel(*args, **kwargs)

基类：peewee.Model

DoesNotExist

BaseModelDoesNotExist 的别名

id = <peewee.AutoField object>

class scylla.database.ProxyIP(*args, **kwargs)

基类：scylla.database.BaseModel

DoesNotExist

ProxyIPDoesNotExist 的别名

city = <peewee.CharField object>

country = <peewee.CharField object>

created_at = <peewee.DateTimeField object>

id = <peewee.AutoField object>

ip = <peewee.CharField object>

is_anonymous = <peewee.BooleanField object>

is_valid = <peewee.BooleanField object>

latency = <peewee.FloatField object>

location = <peewee.CharField object>

organization = <peewee.CharField object>

port = <peewee.IntegerField object>

region = <peewee.CharField object>

stability = <peewee.FloatField object>

updated_at = <peewee.DateTimeField object>

scylla.database.create_connection() → peewee.SqliteDatabase

create a database connection :rtype: SqliteDatabase

scylla.database.create_db_tables()

scylla.jobs module

scylla.jobs.save_ip(p: scylla.database.ProxyIP)

scylla.jobs.validate_proxy_ip(p: scylla.database.ProxyIP)

scylla.loggings module

scylla.scheduler module

class scylla.scheduler.Scheduler

基类：object

feed_providers()

join()

Wait for worker processes and validator threads

start()

Start the scheduler with processes for worker (fetching candidate proxies from different providers), and validator threads for checking whether the fetched proxies are able to use.

stop()

scylla.scheduler.cron_schedule(scheduler, only_once=False)

参数:

• scheduler -- the Scheduler instance
• only_once -- flag for testing

scylla.scheduler.fetch_ips(q: <bound method BaseContext.Queue of <multiprocessing.context.DefaultContext object at 0x7f32ad9fc898>>, validator_queue: <bound method BaseContext.Queue of <multiprocessing.context.DefaultContext object at 0x7f32ad9fc898>>)

scylla.scheduler.validate_ips(q: <bound method BaseContext.Queue of <multiprocessing.context.DefaultContext object at 0x7f32ad9fc898>>, validator_pool: concurrent.futures.thread.ThreadPoolExecutor)

scylla.tcpping module

class scylla.tcpping.Ping(host: str, port: int, timeout=1)

基类：object

get_average() → float

get_maximum() → float

get_minimum() → float

get_success_rate()

ping(count=10, sleep=0.3)

class scylla.tcpping.Socket(family, type_, timeout)

基类：object

close()

connect(host, port=80)

shutdown()

class scylla.tcpping.Timer

基类：object

cost(funcs, args)

start()

stop()

scylla.tcpping.avg(x)

scylla.tcpping.ping(host: str, port: int, count: int = 10, sleep: float = 0.2) -> (<class 'int'>, <class 'float'>)

Ping a server and port with tcp socket :param host: the hostname :param port: the port number :param count: number of connection tries, by default it is 10 :param sleep: length of sleep time in between sequent pings, by default it is 0.3 :return: a tuple for (average_latency, success_rate)

scylla.validator module

class scylla.validator.Validator(host: str, port: int, using_ssl: bool = False)

基类：object

anonymous

latency

meta

success_rate

valid

validate()

validate_latency()

validate_proxy()

scylla.validator.get_current_ip()

scylla.worker module

class scylla.worker.Worker

基类：object

get_html(url: str, render_js: bool = True) → Union[requests_html.HTML, NoneType]

Get html from a specific URL

参数:

• url -- the URL
• render_js -- [whether to render js], defaults to True
• render_js -- bool, optional

返回:

[the HTML string]

返回类型:

str

Module contents