Automated Carousell Alerting System

Check new Carousell listings and sends alerts to Telegram automatically

No longer actively support this project. You can try telegram bot. Leave a comment if you’d like to donate for this to be maintained.

Outcome

Impetus

During this COVID-19 pandemic, I realised most are staying at home and taking the chance to declutter. If you’re like me and one of my best friends who loves to snatch up bargains for cheap thrills, this could be a useful tool for you.

I found multiple resources that teaches you various methods of scraping Carousell. However, most if not all require some level of code writing and understanding.

Here, I will share 2 ways of how you can receive alerts on Telegram.
1) TLDR way: Add alert bot (@caroualerter_bot) on Telegram.
2) DIY way: Deploy your own alert bot.

1) TLDR;

1. On the Telegram app, look for @caroualert_bot.
2. Add an item to receive alerts using this command: /additem <ITEM TO MONITOR>. You will start receiving alerts on that group. (Your TG username and chat_id will be saved. Your mobile number will not be saved.)
3. Stop receive alerts using this command: /deleteitem

2) DIY;

You will deploy your own app that:

1. Periodically polls a Carousell listing page to check for listings
2. Scrapes new listings automatically and filters resellers (optional)
3. Alerts you through Telegram on the new listings

Step 1: Create a Telegram Bot to receive updates

First you have to create your Telegram Bot. You need a Telegram Bot to send you updates. You need to retrieve your (1) Bot Token and (2) chat_id.

Telegram Bot commands

1. Search and send message to BotFather.
2. Type in the commands as seen above.
   (/newbot, <CUSTOM BOT NAME>, <CUSTOM BOT USERNAME>)
3. Create a Telegram Group and invite your Telegram Bot into the group. Your Telegram Bot will use send updates on this group later.
   You use your TG bot so that others will not know what you’re monitoring.
4. Copy and paste your Bot Token and replace it in this link, https://api.telegram.org/bot<YourBOTToken>/getUpdates.
   (i.e.https://api.telegram.org/bot982393737:UNVUNVUV2983293IJDFIJAF82982bjdfh/getUpdates)
5. Visit the link above and you will get a response. Copy and paste the chat_id in the response.
   *(i.e. {“id”:-XXXXXXXXX, ”title”:”asdf”,”type”:”group”,”all_members_are_administrators”:true},”date”:1568560744,”group_chat_created”:true}}]})
   *The minus sign is also part of the chat_id.
6. Ensure you have your (1) Bot Token and (2) chat_id.

Step 2: Deploy App onto cloud platform

Next, you need a Heroku account. You need a Heroku account to host your Bot.

Deploy project on Heroku

1. Sign up for a Heroku account.
2. Go to my Github page, click on “Deploy to Heroku”.

3. Enter the configs accordingly. If you’re still unsure of the configurations, more details can be found here.
I have externalised sensitive configurations so that you can enter it yourself through the Heroku interface. This way, you can be sure that I am not collecting any of your info.

4. As soon as your application have been deployed, you will receive alerts on your Telegram Bot.

5. Optional: You can go to More > View Logs to see what your app is doing behind the scenes!

Releases

22 Sep

• Attached Heroku Add-on LogDNA for better backend logging.

18 Sep

• Refactored code to leverage on Puppeteer cluster which allows for better control of number of pages loaded every single time. Default Puppeteer library launches all item pages at the same time and requires more memory as user base grew. Heroku’s free tier provided 512MB of memory, as soon as this limit was hit, the dyno crashes and starts up every 10 minutes later.

5 Sep

• Automatically removes duplicated listings.

29 Jul

• For @caroualert_bot, migrated from mLab to MongoDB Atlas as mLab will be shutting down its Heroku add-on on November 10, 2020.

7 Jun

• Released refine alerts fulfilling minimum and maximum prices.

16 May

Filter Away Seller option

• Released option to filter away seller with Telegram inline keyboard.

10 May

• Released Telegram bot, @caroualert_bot for users receive alerts.

9 May

• Automatically filters bumpers and spotlighters.
• Optimised to only retrieve document resource types.

Support

Feel free to leave me your suggestions!

I will convey them to my Product Owner, Product Manager, Consultant, Tech Lead, UI Engineer, Release Engineer, and Software Developer.
He will know what to do. :)

For the technical folks…

You can read about technical lessons I’ve learnt, here.