Skip to content

V2.9#776

Open
fadytaher wants to merge 69 commits intodrawrowfly:masterfrom
fadytaher:v2.9
Open

V2.9#776
fadytaher wants to merge 69 commits intodrawrowfly:masterfrom
fadytaher:v2.9

Conversation

@fadytaher
Copy link
Copy Markdown

@fadytaher fadytaher commented Jul 19, 2022

fix post nickname

Fady Taher and others added 30 commits October 16, 2021 01:24
changed schema based on scrape type, simplified signing requests
4611bbf
changed repo
d2dd799
adding build folder
cb9d383
added imageUrl
4a9b531
added imageUrl
0c1a5a4
fixed webvideo url
8fda6da
handling status code
0abbf04
removing logs
e87513b
version marker
2421e41
version marker
d0410b1
versionmarker4
1ae8e99
excluding empty items
bc8a5e6
removing empty objects
7f95bc9
add jar
cf1d2ea
changed userInfo endpoint
03ea4df
removed unrequired logging
828d809
version 3
5ab86ce
added versioning message
18629cf
@fadytaher
Merge pull request #1 from fadytaher/marker.v3
da05e47 
Marker.v3
using tag
ee47012
v2.0.2
1526063
@fadytaher
Merge pull request #2 from fadytaher/marker.v4
c9b9747 
Marker.v4
throwing error if userProfile return empty UserInfo
fb7b113
throwing user doesn't exist if userInfo is empty
a4bee3e
tune error thrown if userPorfile doesn't exist
d9aa5a8
marker v5, 2.0.4
f9a5db2
@fadytaher
Merge pull request #3 from fadytaher/marker.v5
ada027c 
Marker.v5
parse signedURL response
4ea3342
@fadytaher
Merge pull request #4 from fadytaher/marker.v6
c129bbf 
parse signedURL response
v2.1.0
541e72c
Fady Taher and others added 28 commits February 17, 2022 19:14
adding generated url in response items
3d46b24
build project
b4b45fb
adding longUrl to output
6e2b513
@fadytaher
Merge pull request #10 from fadytaher/v2.1.5
6ed1c2e 
V2.1.5
using head instead of get for getVideoLink
05d1e8d
@fadytaher
Merge pull request #11 from fadytaher/v2.1.6
36e08a9 
using head instead of get for getVideoLink
checking requests statuscode
5552a80
@fadytaher
Merge pull request #12 from fadytaher/v2.1.7
a999e47 
checking requests statuscode
checking for status codes and head request
107a1a9
@fadytaher
Merge pull request #13 from fadytaher/v2.1.8
62559dd 
checking for status codes and head request
updated error messages
dc38fd2
@fadytaher
Merge pull request #14 from fadytaher/v2.2
35fd130 
updated error messages
Getting user profile data from HTML content
85ede7e
updated version
671385e
@fadytaher
Merge pull request #15 from fadytaher/v2.3
2669c1d 
V2.3
version 2.4, adding proxy
b69ec64
@fadytaher
Merge pull request #16 from fadytaher/v2.4
b8b7dd3 
version 2.4, adding proxy
logging proxy
9415936
@fadytaher
Merge pull request #17 from fadytaher/v2.5
e302eb1 
logging proxy
added version marker
995e2d1
@fadytaher
Merge pull request #18 from fadytaher/v2.6
498e276 
added version marker
fixed get feed by user id
13f66f4
@fadytaher
Merge pull request #19 from fadytaher/v2.7
be1a2b7 
fixed get feed by user id
working aroud proxy for user-profile
247d25a
added logging
652f90c
added logging
1f9f1e4
@fadytaher
Merge pull request #20 from fadytaher/v2.8
6f073a1 
V2.8
fix post nickname
26f487b
@Nagapetyan
Copy link
Copy Markdown

Nagapetyan commented Jan 10, 2023 •
edited
Loading

doesn't work for me. Build from your repo library local env and docker

Problem 1
tiktok-scraper trend -d -n 100 --session sid_tt xxx

returns nothing.

Problem 2
docker run -v $(pwd):/usr/app/files tiktok-scraper user justinbieber -d -n 5 -s

  • TikTok Scraper Started
    running version is 3.7
    Unexpected token < in JSON at position 0

Problem 3
docker run -v $(pwd):/usr/app/files tiktok-scraper hashtag justinbieber -d -n 5 -s

  • TikTok Scraper Started
    running version is 3.7
    no proxy
    options are :{"jar":{"_jar":{"version":"tough-cookie@2.5.0","storeType":"MemoryCookieStore","rejectPublicSuffixes":true,"cookies":[{"key":"tt_webid_v2","value":"6958744924747484322","domain":"tiktok.com","path":"/","secure":true,"extensions":["hostOnly=false"],"hostOnly":false,"creation":"2023-01-10T18:33:43.062Z","lastAccessed":"2023-01-10T18:33:43.062Z"}]}},"uri":"https://m.tiktok.com/node/share/tag/justinbieber?uniqueId=justinbieber","method":"GET","qs":{"user_agent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_9_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/89.0.4270.182 Safari/537.36"},"headers":{"user-agent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_9_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/89.0.4270.182 Safari/537.36","referer":"https://www.tiktok.com/"},"json":true,"resolveWithFullResponse":true,"followAllRedirects":false,"simple":true,"timeout":10000}
    403 - "<html>\r\n<head><title>403 Forbidden</title></head>\r\n<body>\r\n<center><h1>403 Forbidden</h1></center>\r\n<hr><center>nginx</center>\r\n</body>\r\n</html>\r\n<!-- a padding to disable MSIE and Chrome friendly error page -->\r\n<!-- a padding to disable MSIE and Chrome friendly error page -->\r\n<!-- a padding to disable MSIE and Chrome friendly error page -->\r\n<!-- a padding to disable MSIE and Chrome friendly error page -->\r\n<!-- a padding to disable MSIE and Chrome friendly error page -->\r\n<!-- a padding to disable MSIE and Chrome friendly error page -->\r\n"

Danjordan2319
Danjordan2319 reviewed Mar 14, 2023
View reviewed changes
Copy link
Copy Markdown

@Danjordan2319 Danjordan2319 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

1 more reviewer

@Danjordan2319 Danjordan2319 Danjordan2319 left review comments

Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@fadytaher @Nagapetyan @Danjordan2319