技巧:有什么比较好用的seo排名工具?
优采云 发布时间: 2022-11-29 21:48技巧:有什么比较好用的seo排名工具?
用你的指尖渲染你给我的伤口
自动生成网站检测、网站SEO排查优化任务的一类工具:
" />
【星联SEO管理】可以帮助实现很多SEO功能:站群站点管理与维护、SEO人员协同、优化任务生成、分发与处理、工作考核统计等。
1、输入网站相关信息,只需几分钟,即可完成对所有网站的全面人工智能检测;根据明确的检测结果和待处理任务进行调整和优化,可以延长网站寿命,提高网站排名,增加网站流量和询价获取。
2、系统自动生成SEO优化任务,可人工或实时自动分配给问题网站负责人。比如Starlink检测到的问题的链接(降级,K,404)会第一时间通知负责人。
不仅是短信和邮件,还集成了奇微、钉钉等常用协作工具。
" />
使用本工具相当于找到了一个得力助手,一个前三四的单功能SEO工具
一个是SEO关键词的挖掘和管理工具:
【星矿关键词库管理】关键词挖矿导出维护、关键词管理、关键词预测工具,帮助他们快速准确获取核心关键词 相关 关键词 和长尾词。
还可以自定义几十个标签,外语有20多种,词库管理非常方便。
直观:从代码角度看各类子域名收集工具
class CAInfo(object): def __init__(self, domain): self.domain = domain<br /> def dns_resolve(self): padding_domain = 'www.' + self.domain # loop = asyncio.get_event_loop() loop = asyncio.new_event_loop() asyncio.set_event_loop(loop) resolver = aiodns.DNSResolver(loop=loop) f = resolver.query(padding_domain, 'A') result = loop.run_until_complete(f) return result[0].host<br /> def get_cert_info_by_ip(self, ip): s = socket.socket() s.settimeout(2) base_dir = os.path.dirname(os.path.abspath(__file__)) cert_path = base_dir + '/cacert.pem' connect = ssl.wrap_socket(s, cert_reqs=ssl.CERT_REQUIRED, ca_certs=cert_path) connect.settimeout(2) connect.connect((ip, 443)) cert_data = connect.getpeercert().get('subjectAltName') return cert_data<br /> def get_ca_domain_info(self): domain_list = list() try: ip = self.dns_resolve() cert_data = self.get_cert_info_by_ip(ip) except Exception as e: return domain_list<br /> for domain_info in cert_data: hostname = domain_info[1] if not hostname.startswith('*') and hostname.endswith(self.domain): domain_list.append(hostname)<br /> return domain_list<br /> def get_subdomains(self): subs = list() subdomain_list = self.get_ca_domain_info() for sub in subdomain_list: subs.append(sub[:len(sub) - len(self.domain) - 1]) return subs
查看纯socket实现的dns server
def check(self, dns): logger.info("Checking if DNS server {dns} is available".format(dns=dns)) msg = b'\x5c\x6d\x01\x00\x00\x01\x00\x00\x00\x00\x00\x00\x03www\x05baidu\x03com\x00\x00\x01\x00\x01' sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM) sock.settimeout(3) repeat = { 1: 'first', 2: 'second', 3: 'third' } for i in range(3): logger.info("Sending message to DNS server a {times} time".format(times=repeat[i + 1])) sock.sendto(msg, (dns, 53)) try: sock.recv(4096) break except socket.timeout as e: logger.warning('Failed!') if i == 2: return False return True
根据文本相似度过滤泛解析域名这段代码跨度比较大,下面是简化版
from difflib import SequenceMatcher# RSC ratioself.rsc_ratio = 0.8self.wildcard_html # 获取一个随机子域名的htmlratio = SequenceMatcher(None, html, self.wildcard_html).real_quick_ratio() ratio = round(ratio, 3)if ratio > self.rsc_ratio: # passed logger.debug('{r} RSC ratio: {ratio} (passed) {sub}'.format(r=self.remainder, sub=sub_domain, ratio=ratio))else: # added
其他
ESD只能利用文本相似度来过滤pan-parsing,但是这样会导致机器的内存和CPU暴涨,机器性能小的时候不建议使用。另外,windows下好像没有用ESD,因为最后保存的路径硬编码为/tmp/esd
其他人则认为没有不兼容。解决这个路径后,Windows应该可以使用了。此外
• 解决各DNS服务商判断网络线路出口不一致的问题
• 解决各个DNS服务商缓存时间不一致的问题
• 解决随机 DNS 问题等。
我不知道如何解决这三个问题。也许代码隐藏在某个角落,但我没有找到。
为所有人服务
OneForAll OneForAll 更新很用心。写这篇文章的时候,发现1小时前有一个新的commit。
OneForAll还有很*敏*感*词*,如被动域名搜索、子域爆破、子域接管、端口检测、指纹识别、导出等。
被动搜索
OneForAll集成了很多采集
域名的web接口,每个接口都是一个py文件,py文件最后会基于common/module.py模块类。该类提供了很多常用的方法,比如网页请求、匹配域名、保存结果以及运行时需要的各种方法。比较值得注意的是匹配域名的方法,因为很多web界面返回的格式不一样,每个插件都要重新处理这个格式吗?不是,OneForAll写了一个通用的域名匹配函数,也就是通过正则化对最终的结果进行匹配。
def match_subdomains(domain, html, distinct=True, fuzzy=True): """ Use regexp to match subdomains<br /> :param str domain: main domain :param str html: response html text :param bool distinct: deduplicate results or not (default True) :param bool fuzzy: fuzzy match subdomain or not (default True) :return set/list: result set or list """ logger.log('TRACE', f'Use regexp to match subdomains in the response body') if fuzzy: regexp = r'(?:[a-z0-9](?:[a-z0-9\-]{0,61}[a-z0-9])?\.){0,}' \ + domain.replace('.', r'\.') result = re.findall(regexp, html, re.I) if not result: return set() deal = map(lambda s: s.lower(), result) if distinct: return set(deal) else: return list(deal) else: regexp = r'(?:\>|\"|\'|\=|\,)(?:http\:\/\/|https\:\/\/)?' \ r'(?:[a-z0-9](?:[a-z0-9\-]{0,61}[a-z0-9])?\.){0,}' \ + domain.replace('.', r'\.') result = re.findall(regexp, html, re.I) if not result: return set() regexp = r'(?:http://|https://)' deal = map(lambda s: re.sub(regexp, '', s[1:].lower()), result) if distinct: return set(deal) else: return list(deal)
泛分析处理
域名通过DNS广泛解析时返回的TTL是相同的。泛解析引用一直是域名爆破的一大难题。目前的解决方案是根据不存在的子域记录(md5(domain).domain)获取黑名单IP,对爆破过程的结果进行黑名单过滤。然而,这种广泛的过滤很容易导致误报。比如一般的解析记录是1.1.1.1,但是一个子域也指向1.1.1.1,那么这个子域可能会被黑名单过滤掉。胖同学提到TTL也可以作为黑名单规则的一部分。判断依据是:在权威DNS中,泛解析记录的TTL必须相同。如*敏*感*词*域名记录相同但TTL不同,那么这条记录可以说绝对不是泛解析记录。最终判断代码如下:
// IsPanDNSRecord 是否为泛解析记录func IsPanDNSRecord(record string, ttl uint32) bool { _ttl, ok := panDNSRecords[TrimSuffixPoint(record)] // 若记录不存在于黑名单列表,不是泛解析 // 若记录存在,且与黑名单中的 ttl 不等但都是 60(1min)的倍数,不是泛解析 if !ok || (_ttl != ttl && _ttl%60 == 0 && ttl%60 == 0) { return false } return true}
" />
这个方法好不好,不知道。
喷砂工艺
Brute.py短版爆破过程
wildcard_ips = list() # 泛解析IP列表wildcard_ttl = int() # 泛解析TTL整型值ns_list = query_domain_ns(self.domain) # 查询域名NS记录ns_ip_list = query_domain_ns_a(ns_list) # DNS权威名称服务器对应A记录列表self.enable_wildcard = detect_wildcard(domain, ns_ip_list) # 通过域名指定NS查询是否有泛解析<br />if self.enable_wildcard: wildcard_ips, wildcard_ttl = collect_wildcard_record(domain,ns_ip_list) # 收集泛解析范围,当大部分泛解析记录(80%)达到同一IP出现两次以上,则返回该IP以及TTLns_path = get_nameservers_path(self.enable_wildcard, ns_ip_list)<br /># 生成字典dict_set = self.gen_brute_dict(domain)dict_len = len(dict_set)dict_name = f'generated_subdomains_{domain}_{timestring}.txt'dict_path = temp_dir.joinpath(dict_name)save_brute_dict(dict_path, dict_set)del dict_set<br /># 调用massdns进行扫描output_name = f'resolved_result_{domain}_{timestring}.json'output_path = temp_dir.joinpath(output_name)log_path = result_dir.joinpath('massdns.log')check_dict()logger.log('INFOR', f'Running massdns to brute subdomains')utils.call_massdns(massdns_path, dict_path, ns_path, output_path,log_path, quiet_mode=self.quite,process_num=self.process_num,concurrent_num=self.concurrent_num)
域接管
OneForAll的域名接管,主要是针对一些公共服务的域名接管,根据其指纹识别的内容
[ { "name":"github", "cname":["github.io", "github.map.fastly.net"], "response":["There isn't a GitHub Pages site here.", "For root URLs (like http://example.com/) you must provide an index.html file"] }, { "name":"heroku", "cname":["herokudns.com", "herokussl.com", "herokuapp.com"], "response":["There's nothing here, yet.", "herokucdn.com/error-pages/no-such-app.html", "No such app"] }, { "name":"unbounce", "cname":["unbouncepages.com"], "response":["Sorry, the page you were looking for doesn’t exist.", "The requested URL was not found on this server"] }, { "name":"tumblr", "cname":["tumblr.com"], "response":["There's nothing here.", "Whatever you were looking for doesn't currently exist at this address."] }, { "name":"shopify", "cname":["myshopify.com"], "response":["Sorry, this shop is currently unavailable.", "Only one step left!"] }, { "name":"instapage", "cname":["pageserve.co", "secure.pageserve.co", "https://instapage.com/"], "response":["Looks Like You're Lost","The page you're looking for is no longer available."] }, { "name":"desk", "cname":["desk.com"], "response":["Please try again or try Desk.com free for 14 days.", "Sorry, We Couldn't Find That Page"] }, { "name":"campaignmonitor", "cname":["createsend.com", "name.createsend.com"], "response":["Double check the URL", "Trying to access your account?"] }, { "name":"cargocollective", "cname":["cargocollective.com"], "response":["404 Not Found"] }, { "name":"statuspage", "cname":["statuspage.io"], "response":["Better Status Communication", "You are being <a href=\"https://www.statuspage.io\">redirected"] }, { "name":"amazonaws", "cname":["amazonaws.com"], "response":["NoSuchBucket", "The specified bucket does not exist"] }, { "name":"bitbucket", "cname":["bitbucket.org"], "response":["The page you have requested does not exist","Repository not found"] }, { "name":"smartling", "cname":["smartling.com"], "response":["Domain is not configured"] }, { "name":"acquia", "cname":["acquia.com"], "response":["If you are an Acquia Cloud customer and expect to see your site at this address","The site you are looking for could not be found."] }, { "name":"fastly", "cname":["fastly.net"], "response":["Please check that this domain has been added to a service", "Fastly error: unknown domain"] }, { "name":"pantheon", "cname":["pantheonsite.io"], "response":["The gods are wise", "The gods are wise, but do not know of the site which you seek."] }, { "name":"zendesk", "cname":["zendesk.com"], "response":["Help Center Closed"] }, { "name":"uservoice", "cname":["uservoice.com"], "response":["This UserVoice subdomain is currently available!"] }, { "name":"ghost", "cname":["ghost.io"], "response":["The thing you were looking for is no longer here", "The thing you were looking for is no longer here, or never was"] }, { "name":"pingdom", "cname":["stats.pingdom.com"], "response":["pingdom"] }, { "name":"tilda", "cname":["tilda.ws"], "response":["Domain has been assigned"] }, { "name":"wordpress", "cname":["wordpress.com"], "response":["Do you want to register"] }, { "name":"teamwork", "cname":["teamwork.com"], "response":["Oops - We didn't find your site."] }, { "name":"helpjuice", "cname":["helpjuice.com"], "response":["We could not find what you're looking for."] }, { "name":"helpscout", "cname":["helpscoutdocs.com"], "response":["No settings were found for this company:"] }, { "name":"cargo", "cname":["cargocollective.com"], "response":["If you're moving your domain away from Cargo you must make this configuration through your registrar's DNS control panel."] }, { "name":"feedpress", "cname":["redirect.feedpress.me"], "response":["The feed has not been found."] }, { "name":"surge", "cname":["surge.sh"], "response":["project not found"] }, { "name":"surveygizmo", "cname":["privatedomain.sgizmo.com", "privatedomain.surveygizmo.eu", "privatedomain.sgizmoca.com"], "response":["data-html-name"] }, { "name":"mashery", "cname":["mashery.com"], "response":["Unrecognized domain <strong>"] }, { "name":"intercom", "cname":["custom.intercom.help"], "response":["This page is reserved for artistic dogs.","Uh oh. That page doesn’t exist."] }, { "name":"webflow", "cname":["proxy.webflow.io"], "response":["<p class=\"description\">The page you are looking for doesn't exist or has been moved."] }, { "name":"kajabi", "cname":["endpoint.mykajabi.com"], "response":["The page you were looking for doesn't exist."] }, { "name":"thinkific", "cname":["thinkific.com"], "response":["You may have mistyped the address or the page may have moved."] }, { "name":"tave", "cname":["clientaccess.tave.com"], "response":["Error 404: Page Not Found"] }, { "name":"wishpond", "cname":["wishpond.com"], "response":["https://www.wishpond.com/404?campaign=true"] }, { "name":"aftership", "cname":["aftership.com"], "response":["Oops.
The page you're looking for doesn't exist."] }, { "name":"aha", "cname":["ideas.aha.io"], "response":["There is no portal here ... sending you back to Aha!"] }, { "name":"brightcove", "cname":["brightcovegallery.com", "gallery.video", "bcvp0rtal.com"], "response":["<p class=\"bc-gallery-error-code\">Error Code: 404"] }, { "name":"bigcartel", "cname":["bigcartel.com"], "response":["Oops! We couldn’t find that page."] }, { "name":"activecompaign", "cname":["activehosted.com"], "response":["alt=\"LIGHTTPD - fly light.\""] }, { "name":"compaignmonitor", "cname":["createsend.com"], "response":["Double check the URL or