# common_spider **Repository Path**: liqkjm/common_spider ## Basic Information - **Project Name**: common_spider - **Description**: No description available - **Primary Language**: Unknown - **License**: Not specified - **Default Branch**: master - **Homepage**: None - **GVP Project**: No ## Statistics - **Stars**: 0 - **Forks**: 0 - **Created**: 2019-04-23 - **Last Updated**: 2020-12-17 ## Categories & Tags **Categories**: Uncategorized **Tags**: None ## README ## 环境配置 > python 3.6 > win10 > mysql > pip install -i https://pypi.douban.com/simple pyqt5 > pip install -i https://pypi.douban.com/simple scrapy > pip install -i https://pypi.douban.com/simple pillow > pip install -i https://pypi.douban.com/simple PyMySQL ## 通用爬虫 定制通用爬虫，动态配置爬取规则 1. 只写一份爬虫代码，维护多个网站的爬取规则，其中爬取的字段相同 2. 维护一份代码，爬取多个网站，爬取字段不同 ### 关于购物网站的相同结构 1. 搜索关键字 2. 得到列表页/搜索结果页（通常包括一半需要动态加载的数据） 3. 下一页（/分页接口和总页数）（分页一般和搜索接口相同） 4. 详情页 5. 不同规格 6. 价格接口 ### 需要指定的规则（目前简化版）： 1. 指定开始URL 2. 指定搜索结果中商品URL的位置 3. 指定搜索结果中下一页URL的位置（如果没有下一页的URL，则需要指定分页接口） 4. 指定详情页中需要提取数据的位置和规则 5. 如果有价格接口可以额外指定（如果是通过id查询价格的接口） 6. 查询不同规格 ##### 配置文件，具体规则 三个必须配置：start_urls，rules，item 配置文件详细说明： 1. spider 加不加无所谓 2. website 加不加无所谓 3. default_keyword 默认查询字段，如果在启动爬虫时，未输入关键字，则以该字段为准 4. default_transkey 同上 5. settings 自定义项目配置，按需要添加 5.1 COOKIES_ENABLED 是否开启cookie,1为开启（个别网站需要开启，不然会被ban，比如jumei，一般不用开启，不填即可） 5.2 IMAGES_STORE 指定图片保存路径 5.3 HOST MySQL数据库host 5.4 PORT MySQL数据库port 5.5 USERNAME MySQL数据库username 5.6 PASSWORD MySQL数据库password 5.7 DB_NAME MySQL数据库db_name 5.8 ...具体可查看settings.py，可根据需要重写里面的配置 6. start_urls 爬虫开始页，需要按照格式，指定keyword参数，比如：`https://search.yhd.com/c0-0/k{keyword}/#page=1&sort=1` 7. rules 爬取的规则，包括下一页，详情页，以及不同规格，价格接口规则 7.1 details 详情页规则，即从搜索结果页面中提取商品详情页的规则，这里需要指定allow和restrict_xpath 7.1.1 allow 指定url需要满足的格式，使用正则匹配 7.1.2 restrict_xpath 指定url所在的位置，使用xpath（不懂可百度） 7.2 next 下一页规则，分为两种情况，一个是页面中存在下一页的链接，则和上一个一样，填写allow和restrict_xpath即可，如果不存在，就需要提供分页接口 7.2.1 is_next_url_existed 表明是否存在下一页链接 7.2.2 allow 【如果存在，则补充该规则】筛选url 7.2.3 restrict_xpath 【如果存在，则补充该规则】指定查找的xpath路径 7.2.4 page_total_xpath 【如果不存在，则补充该规则】，这里指定页面中总页码所在的位置 7.2.5 page_url 【如果不存在，则补充该规则】，这里指定分页的url，一般需要包含keyword和page两个参数 7.2.6 page_kwargs 【如果不存在，则补充该规则】，指定keyword和page两个参数的规则（page在代码中是从1-page_total循环的变量，如果分页的接口中也是从1-page_total,则填写"page":"page"即可） ``` "page_total_xpath": "//span[@class=\"fp-text\"]/i/text()", "page_url": "https://search.jd.com/Search?keyword={keyword}&enc=utf-8&page={page}", "page_kwargs": {"keyword": "self.keyword", "page": "page*2-1"}, ``` 7.3 type 爬取商品下的不同规格（比如手机分为32GB和64GB，两者价格不一样，图片可可能不一样） 7.3.1 method 可选xpath或eval两种值，method=xpath,则type.rule需要填写id所在的xpath路径，找出来的结果一般为id列表；如果`method=eval`,则type.rule需要填写可执行代码，满足特殊要求，比如一号店 7.3.2 rule 填写xpath或可执行代码，xpath举例：`"rule": "//*[@id=\"colorItemList\"]/dd/ul/li/@sku"` 7.3.3 url 详情页的url，需要指定一个动态参数，一般为id，比如`https://item.yhd.com/{id}.html` 7.3.4 kwargs 上面url的参数列表，指定参数，type_id是内部代码的变量，即找出来的id列表的循环变量id，如果不需要处理，则直接填写：`"kwargs": {"id": "type_id"}` 7.4 price 价格接口 7.4.1 8. item 从详情页中提取字段的规则 8.1 id 8.1.1 method 这里提供了xpath，css，value，eval四种提取数据的方式 8.1.1 xpath 即xpath路径，例如商品名的提取规则，为`//*[@id=\"productMainName\"]/text()` 8.1.2 css 即css路径（.） 8.1.3 value 给定某个固定值，比如`"website": {"method": "value","value": "一号店"}` 8.1.4 eval 指定可执行代码，满足特殊需要，比如需要从url中提取id，则给定如下代码：`"value": "response.url.split('/')[-1].split('.')[0]"`， 需要说明的是`response`为内部变量，即下载下来的网页对象，`response.url`表示当前url，`response.text`表示网页文本，可以使用的代码还有`response.xpath('')`等 8.1.2 value 即字段的提取规则，可以是四种的任意一种 8.2 website 8.2.1 method 【固定】method=vlaue，直接赋值 8.2.2 value 网站名字 8.3 keyword 8.3.1 method 【固定】method=eval 8.3.2 value 【固定】self.keyword 8.4 trans_key 8.4.1 method 【固定】method=eval 8.4.2 value 【固定】self.trans_key 8.5 name 8.5.1 method 可以根据需要选择xpath，或者eval， 8.5.2 value 指定商品名字的xpath路径，或者能够返回名字的可执行eval代码 8.6 price 【注意，如果可以在页面中直接提取，则填写该字段，如果没有，则不能填写，而应该补充而外的价格接口，或者不补充】 8.6.1 method xpath或者eval 8.6.2 value 提供价格的xpath路径；如果需要额外处理（比如价格接口返回的是json数据，可以使用eval方式， 例：`.loads(response.text)['data'][0]['p']）`，response.text之前已经说过了，为url下载的网页文本，这里即价格接口返回的数据 8.7 image_urls 8.7.1 method 一般需要给出xpath 8.7.2 value 一般需要给出图片url的xpath路径 8.7.3 format 如果提取的图片url不完整，或者想改变尺寸，可以在这里格式化图片url，这里提供的内部变量是`image_url`（单个图片url），可以 针对该变量进行操作，例：`"format": "\"https://img14.360buyimg.com/n1/s500x500_{}\".format(image_url)"`(拼接url) 或`"format": "\"http:\"+image_url.replace('n5','n1')"`（拼接+替换尺寸） 规则示例(yhd.json)： ```json { "spider": "common", "website": "一号店", "default_keyword": "雀巢", "default_trans_key": "Nestle", "settings": { "IMAGES_STORE": "H:/images", "HOST": "localhost", "PORT": "3306", "USERNAME": "root", "PASSWORD": "123456", "DB_NAME": "spider" }, "start_urls": [ "https://search.yhd.com/c0-0/k{keyword}/#page=1&sort=1" ], "rules": { "details": { "allow": "item.yhd.com/\\d+\\.html$", "restrict_xpath": "//a", "comments": "可以通过指定xpath，匹配指定位置的URL，如果不指定则会爬取所有页面符合规则的URL（即不限于列表查询出来的数据，是否和关键字存在出入待验证）" }, "next": { "is_next_url_existed": 1, "allow": ".*", "restrict_xpath": "//a[@class='page_next']" }, "type": { "comments": "一号店有点复杂，首先要找出script的位置，然后使用正则匹配，再转化为json，判断不为空，则提取id...尝试后，无法加判断，代码中统一添加异常处理，针对所有网站", "method": "eval", "rule": "[i['skuId'] for i in json.loads(\"[\"+re.findall(r'availAttrbutes: \\[(.*?)\\],', response.xpath('/html/body/script[1]/text()').extract_first(),re.M | re.S)[0]+\"]\")]", "url": "https://item.yhd.com/{id}.html", "kwargs": {"id": "type_id"} }, "price": { "method": "get", "url": "https://itemapi.yhd.com/getPrices.do?params.area=2_2817_51973_0¶ms.skuIds={id}", "kwargs": {"id": "response.url.split('/')[-1].split('.')[0]"}, "price_rule": "json.loads(response.text)['data'][0]['p']", "comments": "价格使用id查询，price_rule是从价格接口获取的数据中提取价格的规则,这里需要填入字符串形式的可执行代码" } }, "item": { "id": { "method": "eval", "value": "response.url.split('/')[-1].split('.')[0]", "comments": "eval:将字符串转化为可执行代码，这里是从url中提取商品的id，可用于价格接口的请求" }, "name": { "method": "xpath", "value": "//*[@id=\"productMainName\"]/text()" }, "website": { "method": "value", "value": "一号店" }, "keyword": { "method": "eval", "value": "self.keyword", "comments": "固定字段，不用修改" }, "trans_key": { "method": "eval", "value": "self.trans_key", "comments": "固定字段，不用修改" }, "image_urls": { "method": "xpath", "value": "//*[@class=\"detail_main_pic_class\"]/@original_src", "format": "\"https://img14.360buyimg.com/n1/s500x500_{}\".format(image_url)", "comments": "如果url不完整，需要拼接，则提供前缀，如果前缀不为空，则为列表中的每个不完整url拼接前缀，当然可以通过s参数指定图片的大小" } } } ``` //*[@id="J_goodsList"]/ul/li/div/div[1]/a { ":authority": "search.gome.com.cn", ":method": "GET", ":path": "/search?search_mode=normal&reWrite=true&question=%E6%B4%97%E8%A1%A3%E6%9C%BA&searchType=goods&&page=3&type=json&aCnt=0&reWrite=true", ":scheme": "https", "accept": "application/json, text/javascript, */*; q=0.01", "accept-encoding": "gzip, deflate, br", "accept-language": "zh-CN,zh;q=0.9", "cookie": "uid=CjozJ1zQ9cyQNb0NCwLBAg==; cartnum=0_0-1_0; s_ev13=%5B%5B'sem_baidu_cpc_yx_pc21_%25u901A%25u7528%25u8BCD-%25u5730%25u57DF-%25u5168%25u56FD_%25u5730%25u57DF-%25u4E3B%25u8BCD1_%25u6E56%25u5357%25u8D2D%25u7269%25u7F51%25u7AD9'%2C'1557198315573'%5D%5D; compare=; sensorsdata2015jssdkcross=%7B%22distinct_id%22%3A%2216a9040961e4cb-0207626da1e784-58422116-2073600-16a9040961f372%22%2C%22%24device_id%22%3A%2216a9040961e4cb-0207626da1e784-58422116-2073600-16a9040961f372%22%2C%22props%22%3A%7B%22%24latest_traffic_source_type%22%3A%22%E7%9B%B4%E6%8E%A5%E6%B5%81%E9%87%8F%22%2C%22%24latest_referrer%22%3A%22%22%2C%22%24latest_referrer_host%22%3A%22%22%2C%22%24latest_search_keyword%22%3A%22%E6%9C%AA%E5%8F%96%E5%88%B0%E5%80%BC_%E7%9B%B4%E6%8E%A5%E6%89%93%E5%BC%80%22%2C%22_latest_cmpid%22%3A%22sem_baidu_cpc_yx_pc21_%E9%80%9A%E7%94%A8%E8%AF%8D-%E5%9C%B0%E5%9F%9F-%E5%85%A8%E5%9B%BD_%E5%9C%B0%E5%9F%9F-%E4%B8%BB%E8%AF%8D1_%E6%B9%96%E5%8D%97%E8%B4%AD%E7%89%A9%E7%BD%91%E7%AB%99%22%7D%7D; proid120517atg=%5B%229140129042-1130652885%22%2C%22A0006359011-pop8010769376%22%2C%22A0006520651-pop8012473876%22%2C%22A0006520647-pop8012473849%22%2C%22A0006520639-pop8012473803%22%5D; s_cc=true; gpv_p22=no%20value; atgregion=11010200%7C%E5%8C%97%E4%BA%AC%E5%8C%97%E4%BA%AC%E5%B8%82%E6%9C%9D%E9%98%B3%E5%8C%BA%E6%9C%9D%E5%A4%96%E8%A1%97%E9%81%93%7C11010000%7C11000000%7C110102002; DSESSIONID=c197b79bdef947d0a8f986568b0c164f; _idusin=80082039617; route=bbc92a9674ecf2078e553831d27ce2ad; _index_ad=0; gradeId=-1; gpv_pn=no%20value; s_getNewRepeat=1557392037597-Repeat; s_sq=gome-prd%3D%2526pid%253Dhttps%25253A%25252F%25252Fsearch.gome.com.cn%25252Fsearch%25253Fquestion%25253D%252525E6%252525B4%25252597%252525E8%252525A1%252525A3%252525E6%2525259C%252525BA%252526searchType%25253Dgoods%252526search_mode%25253Dnormal%252526reWrite%25253Dtrue%2526oid%253Djavascript%25253Avoid(0)%2526ot%253DA; s_ppv=-%2C20%2C20%2C1506", "referer": "https://search.gome.com.cn/search?question=%E6%B4%97%E8%A1%A3%E6%9C%BA&searchType=goods&search_mode=normal&reWrite=true", "user-agent": "Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/71.0.3578.98 Safari/537.36", "x-requested-with": "XMLHttpRequest" } 2: :authority: search.gome.com.cn :method: GET :path: /search?search_mode=normal&reWrite=true&question=%E6%89%8B%E6%9C%BA&searchType=goods&&page=2&type=json&aCnt=0&reWrite=true :scheme: https accept: application/json, text/javascript, */*; q=0.01 accept-encoding: gzip, deflate, br accept-language: zh-CN,zh;q=0.9 cookie: //uid=CjozJ1zQ9cyQNb0NCwLBAg==; //cartnum=0_0-1_0; //s_ev13=%5B%5B'sem_baidu_cpc_yx_pc21_%25u901A%25u7528%25u8BCD-%25u5730%25u57DF-%25u5168%25u56FD_%25u5730%25u57DF-%25u4E3B%25u8BCD1_%25u6E56%25u5357%25u8D2D%25u7269%25u7F51%25u7AD9'%2C'1557198315573'%5D%5D; //compare=; //sensorsdata2015jssdkcross=%7B%22distinct_id%22%3A%2216a9040961e4cb-0207626da1e784-58422116-2073600-16a9040961f372%22%2C%22%24device_id%22%3A%2216a9040961e4cb-0207626da1e784-58422116-2073600-16a9040961f372%22%2C%22props%22%3A%7B%22%24latest_traffic_source_type%22%3A%22%E7%9B%B4%E6%8E%A5%E6%B5%81%E9%87%8F%22%2C%22%24latest_referrer%22%3A%22%22%2C%22%24latest_referrer_host%22%3A%22%22%2C%22%24latest_search_keyword%22%3A%22%E6%9C%AA%E5%8F%96%E5%88%B0%E5%80%BC_%E7%9B%B4%E6%8E%A5%E6%89%93%E5%BC%80%22%2C%22_latest_cmpid%22%3A%22sem_baidu_cpc_yx_pc21_%E9%80%9A%E7%94%A8%E8%AF%8D-%E5%9C%B0%E5%9F%9F-%E5%85%A8%E5%9B%BD_%E5%9C%B0%E5%9F%9F-%E4%B8%BB%E8%AF%8D1_%E6%B9%96%E5%8D%97%E8%B4%AD%E7%89%A9%E7%BD%91%E7%AB%99%22%7D%7D; //proid120517atg=%5B%229140129042-1130652885%22%2C%22A0006359011-pop8010769376%22%2C%22A0006520651-pop8012473876%22%2C%22A0006520647-pop8012473849%22%2C%22A0006520639-pop8012473803%22%5D; //s_cc=true; //gpv_p22=no%20value; //atgregion=11010200%7C%E5%8C%97%E4%BA%AC%E5%8C%97%E4%BA%AC%E5%B8%82%E6%9C%9D%E9%98%B3%E5%8C%BA%E6%9C%9D%E5%A4%96%E8%A1%97%E9%81%93%7C11010000%7C11000000%7C110102002; //DSESSIONID=c197b79bdef947d0a8f986568b0c164f; //_idusin=80082039617; //route=bbc92a9674ecf2078e553831d27ce2ad; //_index_ad=0; //gradeId=-1; //gpv_pn=no%20value; s_getNewRepeat=1557392548031-Repeat; s_sq=gome-prd%3D%2526pid%253Dhttps%25253A%25252F%25252Fsearch.gome.com.cn%25252Fsearch%25253Fquestion%25253D%252525E6%252525B4%25252597%252525E8%252525A1%252525A3%252525E6%2525259C%252525BA%252526searchType%25253Dgoods%252526search_mode%25253Dnormal%252526reWrite%25253Dtrue%2526oid%253Dfunctiononclick(event)%25257Bjavascript%25253AdoPageNumSearch(1)%25253Breturnfalse%25253B%25257D%2526oidt%253D2%2526ot%253DA; s_ppv=-%2C94%2C20%2C7009 referer: https://search.gome.com.cn/search?question=%E6%89%8B%E6%9C%BA&searchType=goods&search_mode=normal&reWrite=true user-agent: Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/71.0.3578.98 Safari/537.36 x-requested-with: XMLHttpRequest 3: :authority: search.gome.com.cn :method: GET :path: /search?search_mode=normal&reWrite=true&question=%E6%89%8B%E6%9C%BA&searchType=goods&&page=3&type=json&aCnt=0&reWrite=true :scheme: https accept: application/json, text/javascript, */*; q=0.01 accept-encoding: gzip, deflate, br accept-language: zh-CN,zh;q=0.9 cookie: //uid=CjozJ1zQ9cyQNb0NCwLBAg==; //cartnum=0_0-1_0; //s_ev13=%5B%5B'sem_baidu_cpc_yx_pc21_%25u901A%25u7528%25u8BCD-%25u5730%25u57DF-%25u5168%25u56FD_%25u5730%25u57DF-%25u4E3B%25u8BCD1_%25u6E56%25u5357%25u8D2D%25u7269%25u7F51%25u7AD9'%2C'1557198315573'%5D%5D; //compare=; //sensorsdata2015jssdkcross=%7B%22distinct_id%22%3A%2216a9040961e4cb-0207626da1e784-58422116-2073600-16a9040961f372%22%2C%22%24device_id%22%3A%2216a9040961e4cb-0207626da1e784-58422116-2073600-16a9040961f372%22%2C%22props%22%3A%7B%22%24latest_traffic_source_type%22%3A%22%E7%9B%B4%E6%8E%A5%E6%B5%81%E9%87%8F%22%2C%22%24latest_referrer%22%3A%22%22%2C%22%24latest_referrer_host%22%3A%22%22%2C%22%24latest_search_keyword%22%3A%22%E6%9C%AA%E5%8F%96%E5%88%B0%E5%80%BC_%E7%9B%B4%E6%8E%A5%E6%89%93%E5%BC%80%22%2C%22_latest_cmpid%22%3A%22sem_baidu_cpc_yx_pc21_%E9%80%9A%E7%94%A8%E8%AF%8D-%E5%9C%B0%E5%9F%9F-%E5%85%A8%E5%9B%BD_%E5%9C%B0%E5%9F%9F-%E4%B8%BB%E8%AF%8D1_%E6%B9%96%E5%8D%97%E8%B4%AD%E7%89%A9%E7%BD%91%E7%AB%99%22%7D%7D; //proid120517atg=%5B%229140129042-1130652885%22%2C%22A0006359011-pop8010769376%22%2C%22A0006520651-pop8012473876%22%2C%22A0006520647-pop8012473849%22%2C%22A0006520639-pop8012473803%22%5D; //s_cc=true; //gpv_p22=no%20value; //atgregion=11010200%7C%E5%8C%97%E4%BA%AC%E5%8C%97%E4%BA%AC%E5%B8%82%E6%9C%9D%E9%98%B3%E5%8C%BA%E6%9C%9D%E5%A4%96%E8%A1%97%E9%81%93%7C11010000%7C11000000%7C110102002; //DSESSIONID=c197b79bdef947d0a8f986568b0c164f; //_idusin=80082039617; //route=bbc92a9674ecf2078e553831d27ce2ad; //_index_ad=0; //gradeId=-1; //gpv_pn=no%20value; s_getNewRepeat=1557393480158-Repeat; s_sq=gome-prd%3D%2526pid%253Dhttps%25253A%25252F%25252Fsearch.gome.com.cn%25252Fsearch%25253Fquestion%25253D%252525E6%25252589%2525258B%252525E6%2525259C%252525BA%252526searchType%25253Dgoods%252526search_mode%25253Dnormal%252526reWrite%25253Dtrue%2526oid%253Djavascript%25253Avoid(0)%2526ot%253DA; s_ppv=-%2C16%2C16%2C1165 referer: https://search.gome.com.cn/search?question=%E6%89%8B%E6%9C%BA&searchType=goods&search_mode=normal&reWrite=true user-agent: Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/71.0.3578.98 Safari/537.36 x-requested-with: XMLHttpRequest *手机*： 2: uid=CjozJ1zQ9cyQNb0NCwLBAg==; cartnum=0_0-1_0; compare=; atgregion=11010200%7C%E5%8C%97%E4%BA%AC%E5%8C%97%E4%BA%AC%E5%B8%82%E6%9C%9D%E9%98%B3%E5%8C%BA%E6%9C%9D%E5%A4%96%E8%A1%97%E9%81%93%7C11010000%7C11000000%7C110102002; s_cc=true; gpv_p22=no%20value; DSESSIONID=54340c2ec771458eb4e34c77d7f92376; _idusin=80076334886; s_ev13=%5B%5B'sem_baidu_cpc_yx_pc21_%25u901A%25u7528%25u8BCD-%25u5730%25u57DF-%25u5168%25u56FD_%25u5730%25u57DF-%25u4E3B%25u8BCD1_%25u6E56%25u5357%25u8D2D%25u7269%25u7F51%25u7AD9'%2C'1557198315573'%5D%2C%5B'sem_baidu_pinpai_yx_pc_bt'%2C'1557458156393'%5D%5D; route=2dbe4187f565101d7a8af62de1b9dab0; gradeId=-1; proid120517atg=%5B%229134520865-1123460337%22%2C%229140129042-1130652885%22%2C%22A0006359011-pop8010769376%22%2C%22A0006520651-pop8012473876%22%2C%22A0006520647-pop8012473849%22%2C%22A0006520639-pop8012473803%22%5D; sensorsdata2015jssdkcross=%7B%22distinct_id%22%3A%2216a9040961e4cb-0207626da1e784-58422116-2073600-16a9040961f372%22%2C%22%24device_id%22%3A%2216a9040961e4cb-0207626da1e784-58422116-2073600-16a9040961f372%22%2C%22props%22%3A%7B%22%24latest_traffic_source_type%22%3A%22%E7%9B%B4%E6%8E%A5%E6%B5%81%E9%87%8F%22%2C%22%24latest_referrer%22%3A%22%22%2C%22%24latest_referrer_host%22%3A%22%22%2C%22%24latest_search_keyword%22%3A%22%E6%9C%AA%E5%8F%96%E5%88%B0%E5%80%BC_%E7%9B%B4%E6%8E%A5%E6%89%93%E5%BC%80%22%2C%22_latest_cmpid%22%3A%22sem_baidu_pinpai_yx_pc_bt%22%7D%7D; gpv_pn=no%20value; s_sq=gome-prd%3D%2526pid%253Dhttps%25253A%25252F%25252Fsearch.gome.com.cn%25252Fsearch%25253Fquestion%25253D%252525E6%252525B4%25252597%252525E8%252525A1%252525A3%252525E6%2525259C%252525BA%252526searchType%25253Dgoods%252526search_mode%25253Dnormal%252526reWrite%25253Dtrue%2526oid%253Djavascript%25253Avoid(0)%2526ot%253DA; s_ppv=-%2C15%2C15%2C1040; s_getNewRepeat=1557458414441-Repeat; _index_ad=1 3: uid=CjozJ1zQ9cyQNb0NCwLBAg==; cartnum=0_0-1_0; compare=; atgregion=11010200%7C%E5%8C%97%E4%BA%AC%E5%8C%97%E4%BA%AC%E5%B8%82%E6%9C%9D%E9%98%B3%E5%8C%BA%E6%9C%9D%E5%A4%96%E8%A1%97%E9%81%93%7C11010000%7C11000000%7C110102002; s_cc=true; gpv_p22=no%20value; DSESSIONID=54340c2ec771458eb4e34c77d7f92376; _idusin=80076334886; s_ev13=%5B%5B'sem_baidu_cpc_yx_pc21_%25u901A%25u7528%25u8BCD-%25u5730%25u57DF-%25u5168%25u56FD_%25u5730%25u57DF-%25u4E3B%25u8BCD1_%25u6E56%25u5357%25u8D2D%25u7269%25u7F51%25u7AD9'%2C'1557198315573'%5D%2C%5B'sem_baidu_pinpai_yx_pc_bt'%2C'1557458156393'%5D%5D; route=2dbe4187f565101d7a8af62de1b9dab0; gradeId=-1; proid120517atg=%5B%229134520865-1123460337%22%2C%229140129042-1130652885%22%2C%22A0006359011-pop8010769376%22%2C%22A0006520651-pop8012473876%22%2C%22A0006520647-pop8012473849%22%2C%22A0006520639-pop8012473803%22%5D; sensorsdata2015jssdkcross=%7B%22distinct_id%22%3A%2216a9040961e4cb-0207626da1e784-58422116-2073600-16a9040961f372%22%2C%22%24device_id%22%3A%2216a9040961e4cb-0207626da1e784-58422116-2073600-16a9040961f372%22%2C%22props%22%3A%7B%22%24latest_traffic_source_type%22%3A%22%E7%9B%B4%E6%8E%A5%E6%B5%81%E9%87%8F%22%2C%22%24latest_referrer%22%3A%22%22%2C%22%24latest_referrer_host%22%3A%22%22%2C%22%24latest_search_keyword%22%3A%22%E6%9C%AA%E5%8F%96%E5%88%B0%E5%80%BC_%E7%9B%B4%E6%8E%A5%E6%89%93%E5%BC%80%22%2C%22_latest_cmpid%22%3A%22sem_baidu_pinpai_yx_pc_bt%22%7D%7D; gpv_pn=no%20value; _index_ad=1; s_getNewRepeat=1557459323852-Repeat; s_sq=gome-prd%3D%2526pid%253Dhttps%25253A%25252F%25252Fsearch.gome.com.cn%25252Fsearch%25253Fquestion%25253D%252525E6%25252589%2525258B%252525E6%2525259C%252525BA%252526searchType%25253Dgoods%252526search_mode%25253Dnormal%252526reWrite%25253Dtrue%2526oid%253Djavascript%25253Avoid(0)%2526ot%253DA; s_ppv=-%2C14%2C14%2C949 # 表 ```mysql CREATE TABLE `common_sku` ( `id` varchar(45) NOT NULL, `website` varchar(45) NOT NULL, `keyword` varchar(45) NOT NULL, `trans_key` varchar(45) NOT NULL, `name` varchar(100) NOT NULL, `price` varchar(45) DEFAULT NULL, `label` varchar(45) DEFAULT NULL, `image_urls` varchar(1000) NOT NULL, PRIMARY KEY (`id`) ) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_0900_ai_ci ```