question

InactiveAutoUser avatar image
0 Likes"
InactiveAutoUser asked

Crawling through findItemsAdvanced pages results in duplicates?

I am using eBay's findItemsAdvanced API call in order to download all items from a given seller. Through my testing, I've noticed that when I hit larger sellers, say with 8000+ items, my results are iffy.

 

Firstly, the number of items I end up saving (using the itemID as a KEY so any duplicates are weeded out) ends up being 1-2% under the actual number reported by eBay (both on their website and through the findItemsAdvanced call). Secondly, I logged the item IDs each page I downloaded returned, and I see between 1-9 items overlapped between pages, and not only that, in both directions (that is, say page 28 overlapped page 29 by 5 items, and overlapped page 27 by 1 item). The sellers I'm working on have fixed price items and I can verify no new items were sold/removed/added during the time I initiated the download and finished it.  

 

I download items via a threaded application, pinging eBay's API rapidly for up to 5 pages per thread (among 20 threads) all at once.  I haven't been testing for any sellers of 10,000 items as I know this to be a special case/limitation of the findItemsAdvanced call. I keep the ordering as EndTimeSoonest and do 100 items per page. 

 

What I do know is that if I were to compare the number of duplicates to the actual number of unique items, the # of duplicates + # of unique items more or less equals the number reported by eBay. 

 

So my question is, is there a fault in my code causing this issue or a limitation of the eBay API? Is there no guarantee that each page will contain no duplicates when compared to a page before/after (depending on sorting I guess)? I realize I can download a page and try to page through the API by setting a EndTimeFrom filter based on the last item I downloaded, but this isn't achievable at the scale and speed I want to achieve. I lose the capability of threading in that scenario.

 

Please let me know if this is just eBay's API that I'll have to accept/work around or an issue with my idea/code.

 

Thank you in advanced!

finditemsadvancedduplicates
10 |600 characters needed characters left characters exceeded

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

1 Answer

· Write an Answer
dockguys avatar image
0 Likes"
dockguys answered
I am seeing the same issue with an account with 7000 items. Has anyone found a work around?
10 |600 characters needed characters left characters exceeded

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

Write an Answer

Hint: Notify or tag a user in this post by typing @username.

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.