Software Engineer
At Daltix (Full-time), in Lisbon, Portugal
Expires at: 2019-11-10
We are looking for talented profiles to help build and maintain the distributed data collection system that is at the heart of our business. You'll be in the front-lines, facing massive (but interesting!) challenges as we try to scrape all retail data available.
We are a data-driven company which collects and processes more than 600GB of raw data (HTML) daily. We leverage big data technologies such as Serverless, Spark on AWS EMR to crunch these volumes of data and make it queryable.
In this role, you will ensure that our data collection engine, which consists of distributed web crawlers, is state of the art and ahead of our competition. You will ensure that we can scrape any webshop, no matter the ban-detection that has been put in place. Then, it will be important that proper monitoring tools are in place. We are currently scraping 60 sites and your goal is to at least triple that without losing completeness and quality.
Responsibilities
Developing & maintaining web crawlers using Python & JavaScript.
Designing & developing internal tools and frameworks.
Designing & developing tools to automate testing & QA.
Building & maintaining distributed data collection systems on AWS.
Guarantee data integrity and quality by extending our logging, monitoring and outlier detection systems.ng
Main requirements
Knowledge of Computer Engineering (through formal education or other means)
Strong programming experience with one or more languages: Python (preferred), Javascript, C#, Java, Scala, C/C++
Good understanding REST API's, databases and SQL
Linux command-line experience
Highly proficient in spoken and written English
You are passionate about software engineering
Nice to have
Strong experience with both Python & JavaScript
Experience working on top of Amazon Web Services
Excellent coding skills and deep knowledge of web technologies
Excellent problem-solving capabilities and a critical mindset
You get energy from working in a highly complex and challenging startup environment with a high tech product
DevOps & automation (Terraform, Ansible)
Data analysis using Pandas (Python)
Perks
Work with the latest tech stack
We're a quickly growing company => you're personal growth can be huge too.
Health Benefits – Comprehensive coverage for medical
Meal allowance – Monthly meal card along with a fully stocked kitchen with enough coffee and fruits, along with monthly team drinks and dinners
Work-Life Balance – We trust you to know your schedule and work when you feel most productive
Learning and Development – Attend meet-up, conferences, and events that interest you and benefit your personal and career growth.
Apply for this position
---------------------------------------------------------------------------
Visit this link to stop these emails: http://zpr.io/gkQ3Q
Post a Comment