This is just a short note to specify that the list of FREE datasets is updated for 2020. There are 50+ sites and links to the newly released Google Dataset search engine. So, have fun exploring these data repositories to master programming, create stunning visualizations and build your own unique project portfolios.
Some starter projects with these datafiles are available on the Projects page, using R-programming.
The first month of the new decade is almost at an end. It’s also “job-hunting” time when students start looking for internships and employees think about switching roles and companies, in search of better salaries and opportunities. If you fall into one of these categories, then here are the Top 10 skills your resume absolutely needs to include, to get noticed by employers and land your dream job.
I looked at 200 job descriptions for jobs posted on LinkedIn in 7 major US/Canada cities – San Francisco, Seattle, Chicago, New York, Philadelphia, Atlanta, Toronto. Let’s face it – LinkedIn is the go-to platform for job seekers and recruiters. So looking at any other site seems a waste of time.
The job listings included many of the top Global brands in tech (Microsoft, Amazon, etc.), product (AirBnb, Uber, Visa), consulting (Deloitte, Accenture), banks (JP Morgan, Capital One) and so on. I only considered jobs with the title “Data scientist” or “Data Analyst”, with 150+ in the former. It took a while, but doing this manually also allowed me to exclude repetitive postings, since some companies post same role for multiple locations.
Ultimately, this allowed me to quickly identify patterns and
repeated skills, which I am presenting in this blogpost.
I’ve categorized the skills into 2 parts: Core and Advanced. Core skills are the absolute minimum you should have, recruiters and automated job application systems will simply disqualify you without them. Advanced skills are those “preferred” competencies that make you look more valuable as a candidate, so make sure to highlight them with examples on your resume. So, if you are trying to transition to a career in Data Science, then I would highly recommend learning these first, and then jumping into the others. Needless to say, everyone working (or entering) this field needs to have a portfolio of projects.
Disclaimer – having all the 10 skills does NOT guarantee a job but vastly improves your chances. You’ll still need to do some legwork, to get considered and my book “Data Science Jobs” can help you shorten this process. The book is also on SALE for $0.99 this weekend, Jan 25th to Jan 28th, at a 92% discount.
 Programming (R/Python): This is a no-brainer, you need to be an expert in either R/Python. Some jobs will list SAS or other obscure languages, but R or Python was a constant and mandatory requirement in 100% of all the jobs I parsed.
I am not going to argue the merits of one over the other in this post, but I will emphasize that R is still very much a in-demand skill. Plus, for most entry level roles, a candidate with only Python is not going to be considered more favorably (or declined!) over someone who knows only R. In fact, at my current and previous 2 roles, R-programming was the preferred language of choice. If you’d like to know my true views on the R vs Python debate, read this post.
 SQL: Most colleges and bootcamps do not teach this, but it is inordinately valuable. You cannot find insights without data, and 99% of companies predominantly use SQL databases of some kind. Fancy stuff like MongoDb, NoSQL or Hadoop are excellent keywords to add to your bio, but SQL is the baseline. You don’t need to know stored procedures or admin level expertise, but please learn basics of SQL for pulling in data with filters and optimizing table joins. SQL is mandatory to thrive as a data scientist.
 Basic math & Stats: By this I mean basic high-school stuff, like calculating confidence intervals and profit-loss calculations. If you cannot distinguish between mean and median, then no self-respecting manager will trust your numbers, or believe your insights have excluded those pesky outliers. Profits, incremental benefits in $ are other useful formulae you should know too, so brush up on your business math.
 Machine Learning Algorithms: Knowing to code algorithms is expected, but so is knowing the logic behind them. If you cannot explain it in plain English, you really don’t know what you are talking about!
 Data Visualization: Tableau is the preferred technology, although I’ve seen people find success with Excel charts ( Excel will never die! ) and R libraries, too. However, I definitely see Tableau dominating everything else in the coming years.
 Communication skills: A picture is worth 1000 words; and being able to present data in meaningful, concise ways is crucial. Too many newbies get lost in the analysis itself, or hyper-focused on their beautiful code. Most managers want to see recommendations and insights that they can apply in practice! So being able to think like a “consultant” is crucial whether you are entry-level or the lead data scientist.
Good presentation skills (written and verbal) are important, more so for any dashboards or visualization reports, and I don’t mean color palettes or chart-types. Instead, make sure your dashboards are not “data-vomit”, a very practical (and apt!) term coined by Avinash Kaushik. If users cannot make head or tail of the dashboard without your handholding, or if the most important take-away is not obvious within 5 seconds, then you’ve done a poor job.
 Cloud services: Most companies have moved databases to AWS/Azure, and many are implementing production models in the cloud. So, learn those basics about Docker, containers, and deploying your models and code to the cloud. This is still a niche skill, so having it will definitely help you stand apart as most companies make the move towards automation.
 Software engineering: You don’t need to become a software engineer but knowing basic architecture and data flow Qs will help you troubleshoot better, write better code that is easily moved to production. Some Qs to start – what is the data about, where (all) is it coming from? Learn about scheduler jobs and report automation, these have helped me automate the most boring repetitive tasks and look like a superstar to my managers! The infrastructure teams do extremely valuable work (keeping things running smoothly) so learn about “rules” and expectations, and make sure your code conforms to everything. I always do, and my requests are treated much better! 😉
 Automated ML: This is slowly getting popular, as companies try to cut costs and improve efficiencies with automation. H20.ai and DataRobot are just 2 names off my head, but there are many more vendors in the market. If possible, learn how to work with those, as they can reduce your time for analysis and speed up production deployment. They won’t replace good data scientists, but they do magnify the disparity between someone who is mindless copy/pasting code and the truly efficient data scientists. So make sure your “core” skills are impeccable.
 Domain expertise: Nothing beats experience, but even if you are new to the company (or field) learn as much as you can from senior colleagues and partner teams. Find out the “why/how/what” Qs – who is using the analysis results, why do they truly want it? How will it be applied? How does it save the company money or increase profits? How can I do it faster while maintaining accuracy, and also adding to the bottom line? What metric does the end user (or my manager) really care about?
As Machine learning software add more automation and features, this blend of technology and domain expertise will ensure you are never a casualty of layoffs or cost-cutting! I’ve put this at the end, but really you should be thinking about this from DAY ONE!
For example, my current role involves models for credit card fraud prediction. However, once I learned the end-to-end process of card customer lifecycle (incoming application, review, collections, payments, etc.) my models have become much better. Plus, I have deeper understanding of Fair banking and privacy laws which can prevent many demographic variables from being used in modes. Similarly, a friend working in the petrochemical industry realized that his boss cared more about preventing true negatives (Overlooking or NOT maintaining end-of-life or faulty sensors that can potentially cause leaks or explosions ) than false positives (unnecessary maintenance for good sensors), even though both models can give you similar accuracy.
So build these skills, and see your career and salary potential sky-rocket in 2020!
Just 5 days left to KubeCon + CloudNativeCon North America! 🙂 I am quite excited to finally attend this awesome conference and the chance to visit sunny San Diego! 🙂 Whether you are a first time attendee as well, or just looking to get your money’s worth from the conference here is a list of to-dos to make the most of this experience.
If you have not heard about KubeCon, then this is a conference aimed at Kubernetes and related container technologies, a way to get software applications running with cloud services. This is an entire ecosystem, and in the next few years will change software infrastructure concepts for all companies. Myriad companies including Uber, Google, Shopify, JPMorgan are already on board and deploying using these new methods.
These technologies are also a huge part of how machine learning models and AI applications are implemented successfully and at scale, which is why I (as owner of this datascience blog) got interested in Kubernetes. If you’ve run machine learning models using cloud services, you might have also used some of these tools without ever being aware of it.
This is (obviously) my first time attending this conference and visiting the city, so I had tons of Qs and thoughts. The amazing list of speakers and conference tracks also make it hard to choose which sessions to attend. Thankfully, I was able to get some excellent advice from the dedicated Slack channels for the conference and past attendees.
Since the countdown clock has started, I’ve summarized the tips for others, so you can make the most of this experience.
1. Get on Slack
I am so thankful to Wendi West, Paris Pittman and other moderators in the Slack channels for patiently answering questions, hotel recommendations, sending event reminders and building some great vibes for the conference! the organizers for this conference .
I found a lot of useful information on the channel specific to the Diversity scholarship recipients, followed by the events channel. If you still have last minute Qs, then post on this channel or DM the organizers.
The Slack channels are great to connect with folks before the event, so you have some familiar faces to meet at the conference.
By now everyone should have created an itinery for themselves. If not please use the “sched” app with the following workspace url https://kccncna19.sched.com/
Note that you should have one broad agenda for the conference – either info for attempting a certification, networking for a job, discuss case studies so you can apply concepts at your job, or something else. This will allow you to make better selections without feeling overwhelmed by the sheer number of (fantastic) choices!
For me, the main goals is understand how Kubernetes is deployed for machine learning & AI projects. Since I work for a bank, security issues and migration of legacy/mainframe software into cloud services are also relevant topics, as are case studies. Having this theme allowed me to quickly decide and create a meaningful list of sessions to attend. I also hope to network with 25+ new people, a goal that should be quite easy in a conference with 8000+ attendees and spanning 5 days.
PRO TIP: Couple of past attendees advised not to over-schedule and to look at room locations. Although the conference is mainly happening at the San Diego convention center, there are some sessions that are being held at the Marriot and other hotels. So look them up and make sure you have enough time to walk to different venues.
Plus, if a session is quite interesting, you might want to hang back and chat with the speaker or ask additional clarifications. This might cause you to miss the next session, so design your schedule carefully.
Being part of a distince Diversity scholarship Slack channel means that I’ve already connected with 5-10 other recipients. After all these online discussions, it will great to meet these talented and ambitious folks in person!
Most past attendees have emphatically stated that folks attending this conference are very generous with their time, so please make the most of their expertise and knowledge.
Dont be shy! Speak up.
Speakers are amazing, and human too! So feel free to say hello after the session, and ask follow up Qs or just thank them for an interesting discussion.
For those who are extremely nervous about networking, here is a unique tip that someone told me years ago. Pick a color and talk to at least 5 people wearing clothes in that color. This might seem crazy, but it is a very practical way of overcoming self-bias and prejudices and talking to people we would not normally approach (feeling shy, out of place or other reasons) I’ve used it at other events and conferences and made some fabulous connections!
Use LinkedIn app and connect immediately. If you met someone interesting, then send them an invite during the conversation itself. No one ever says no, and if you wait you will forget to followup, either because you forgot their name, used the wrong spelling or misplaced their business card. Plus, at such large conferences it is terribly hard to keep track of all the people you meet. I used this at the Philly AI conference very successfully, and can’t wait to connect with folks at KubeCon too!
During the conference:
Keep a notepad handy for broad keywords and ideas that are directly applicable to your role (and conference goal) .
List the session date, speaker name and time. This will help you look it up later, esp as I’ve heard many speakers post their ppt and videos after the conference.
Tweet! Use the tags #KubeCon #CloudNativeCon and #DiversityScholarship.
Connect with people on LinkedIn (reiterating from above)
Check out the sponsored coffee/breakfast sessions and after hour meetups; I’ve heard they are amazing, as are the “lightning talks” post 5 pm.
Attend the sponsor expo and booths. Apart from the cool swag (tees, pens, stickers, etc) you will get to see some interesting demos, hobnob with folks from companies both large and small. Basically everyone from startups to large enterprises like Microsoft and Palo Alto, and everything in between. Great way to learn what’s happening in this space – you might even get your own unicorn startup idea! 🙂
Reconnect with the folks you’ve met on LinkedIn.
Add a blog post summarizing info you’ve learnt and takeaways from the conference. Everyone has a unique perspective, so don’t feel as if everything has already been said! Ideally do this within a week, when you are still fresh with your ideas. Remember to use the hashtags.
Add pictures from the event on LinkedIn. Make sure to tag your new friends too!
If possible, present a brown bag or session to your team (or group) at office. This is a great way of disseminating information to others who could not attend, improve your public speaking skills and also score some brownie points for your next employee appraisal! Win-win all around.
Use what you’ve learnt. Even if it is just a little portion!
I had the opportunity to attend the third annual PHLAI 2019 conference on Artificial Intelligence and Machine Learning. Organized by Comcast, this was an amazing full-day event (on Aug 15th) with a theme of using AI to improve customer experience.
For those who are not in the Greater Philadelphia area, I have to point out that most AI/Machine learning or even tech conferences typically happen either on the West Coast or in NYC. So the cost of travel/lodging can quickly add up if you want to attend as an individual.
Always been a point of contention to me that there aren’t any interesting conferences held in the Philly area. After all, we are home to a large number of startups (RJMetrics, Blackfynn, Stitch, Seer Interactive), Fortune500 employers like Nasdaq, Comcast, Vanguard and most of the big banks (JPMorgan, Barclays, etc.)
So here is a big shoutout to Comcast (esp. Ami Ehlenberger) for making the magic happen in Philly!
Practical tips on how to use AI to create individual customization, especially to improve conversions on business websites.
For getting most ROI on AI investments, begin with smaller AI projects and deploy/iterate. Most projects fail because they aim at initiating a massive overhaul that take so long to move to production, that half the assumptions and data become invalid. Divide and conquer.
Don’t use AI just for the sake of it.
Real time data is crucial. 360 degrees view by aggregating multiple sources will help to drive maximum effectiveness, especially in banking and finance.
He also presented some interesting examples of AI projects that are already in use today in various domains:
Early screening of autism & parkinson using front camera of phone. Apparently the typing/ tapping on phone is different for such patients and when combined with facial recognition technology, the false positives have been very low.
Drones in insurance. One company used drones to take pictures to assess damage after hurricane Harvey and proactively filed claims on behalf of their customers. This meant that the papaperwork and claim money was released for affected customers. Home repair contractors are in heavy demand after such natural disaster but the customers from this company had no trouble getting help because everything was already set up! What better example of extraordinary customer service than helping your customers re-build their life and homes? WOW!
Drone taxi in Dubai. Basically a driverless helicopter. Scary yet fascinating!
VR glasses for shopping. Allows customers to interact with different products and choose color/ design/ style. Ultimate customization without the hassles of carrying gigantic amounts of inventory.
This was presented by Piers Lingle & David Monnerat from Comcast. They spoke about the AI projects implemented at Comcast including projects to proactively present options to customers that increase customer experience, reduce call center volumes and complaints. I loved the idea of using automation to present 2-way dialog using multiple means of communication. The also spoke about chatbots and strategies that help chatbots learn at scale.
Some other salient points:
Automation is a part of the experience, not an add-on or an extra bother
Voice remote for Comcast TV users allows customers to fast forward for commercial breaks. Cool! 🙂
Think beyond NLP. AI works better in troubleshooting tasks when customers are struggling with describing problems, for example troubleshooting internet connectivity issues.
Measure from the beginning . Measure the right thing. Easy to say, hard to do.
Done correctly, AI does seem like magic. However, it takes a lot of work, science and complexity. So appreciate it! 🙂 Above all, trust the tech experts hired for this role instead of pigeoning them with hyper-narrow constraints but high expectations.
The afternoon had 3 different tracks of speaker sessions with 3 talks each – Product, Customer Service & Real-World Experience. Attendees were allowed to attend talks from all 3 tracks which was what I did. The speaker lineup was so spectacular, I noticed a lot of folks being spoilt for choice and scratching their heads after every talk, on where to go! 🙂
I am only writing about the ones I attended. However, I did receive positive feedback overall for all the other sessions too! (No offense to the folks who presented but are not mentioned on here! )
Workshop by Datarobot.
This is an interesting software that helps you explore data, automate data cleansing and most important test by running multiple models in parallet. So you could test with hundreds of models and algorithms. The workshop was presented by the VP of DataScience, Gourab De and was extremely appealing.
It does have features for tweaking parameters, and excellent documentation so personally I think it would greatly augment the work of an experienced data scientists. Given the shortage of talent, tools like these would allow smaller teams to become faster and more efficient.
The availability of logs is another fabulous feature for banks (since I work in one) due to the stringent regulatory requirements around compliance, ethics and model governance.
Chaos Engineering & Machine Learning
Presented by Sudhir Borra of Comcast itself. The talk was mainly about the concept of chaos engineering; used to test resiliency of systems. This is quite crucial for serverless AI and automated CI/CD pipelines when models are tested and deployed in the cloud. This was quite technical yet practical to teams and companies looking to implement robust AI systems within their organizations.
Data Stories Tableau Session
Presented by Vidya Setlur of Tableau, who spoke of some interesting features with Tableau. For example, did you know Tableau gives logical color palettes where it makes sense, rather than to default colors? For example for a dataviz on vegetables, brocolli is assigned color green, whereas carrots would be assigned orange. She also spoke about “stroop” effect, which I urge every data scientist or Tableau developer to look up and understand.
There were also some interesting Qs about expanding the natural langauge questions to be activated via voice. The answer was that it is currently not available because the folks who would truly find this useful are senior executives who would possibly only use it over phone (not desktop) and hence also require touchscreen capabilities for using drill-down features.
This was a perspective I had not thought about, so I was quite impressed at how far the company has thought about features users might want far into the future. Like Apple, Tableau’s elegant design, ease of use and intuitive features make it a class apart in terms of functionality and customer experience.
The Humane Machine: Building AI Systems with Emotional Intelligence
He also spoke of LIWC library – linguistic inquiry and word count and how to apply those on natural langauge data to gauge sentiments, similar to the Plutchiks wheel of emotions. Also on how to use these for creating more “sentient” chatbots and applications, since the context matters as much as words when we (humans) communicate. For example, how some cultures use words literally, whereas others could use it to mean the exact opposite!
He also discussed ethics in AI, which naturally was a “hot” topic and sparked multiple questions and discussions.
Handson Labs – H20 – Automated AI/ML
I’d heard many rave reviews about this product, so it was wonderful to be able to attend a handson workshop on how to use the tool, available features and check effectiveness of the models it can generate. Personally I found it to have just the right mix of automated model generation (great for newbies) and model customization (for advanced users).
The conference included two rounds of networking time, once with free breakfast, and second at 5:15 pm in a formal networking reception. Both were great opportunities to meet with other attendees and speakers.
I was not attending with any aim for job hunting, but I did see a list of open roles at Comcast. Plus, I did see folks talking about roles in their org, and describing their work, so certainly the event was an excellent venue to have informational interviews and reach out for potential collaboration.
Not to mention the new contacts will be useful in the future, should I choose to look.
I met a wide variety of folks – from students to software developers to product leads and even senior engineering leaders who wanted to learn best practices on how best to get ROI on future AI investments. There were folks who were just interested in learning more about AI, some looking for their next big idea and everyone in between. It was not just tech folks who attended, and folks were attending from domains like adtech, marketing, automobiles, healthcare and of course banking.
Amazing to see the turnout and community. Overall, I think job seekers should attend such events!
Oh, and I have to mention the gorgeous Comcast building – amazing working space and the central atrium was both futuristic and functional. I definitely regretted not taking the Comcast tour! (In fairness, the workshop I attended was fabulous too! )
Of course there was also time to chitchat between sessions and I am thankful to everyone who interacted with me! Thanks for your sparkling conversations.
Focus on the customer and the end goal. Technology is only a tool to enable better outcomes and efficiencies. AI tools have matured, so make full use of it.
For the best return and impacts from AI projects, quick action is key. Choose a specific usecase, choose a metric to evaluate success and then proceed. If the model works, BRAVO! If they do not, you get valuable feedback on your initial assumptions, so you can make better decisions in the future.
Clean, reliable data is paramount. Garbage in, garbage out!
Collaboration is important. You do need to get the blessings from multiple stakeholders like IT, business, product, customer service, etc. But do not let collaboration equate to decision paralysis.
Done right, AI can be magical and live up to the hype. The critical component is getting started.
Data and user preferences change quickly, so think about automating model deployments and/or implementing CI/CD using technologies like kubernetes or the myriad tools and software available on the market to get to production quicker.
There is a huge talent shortage of experienced machine learning and artificial intelligence developers, so think about “upskilling” existing employees who already know the business and customer pain points.
/ JOURNEYOFANALYTICS / Comments Off on DataScience Portfolio Ideas for Students & Beginners
A lot has been written on the importance of a portfolio if you are looking for a DataScience role. Ideally, you should document your learning journey so that you can reuse code, write well-documented code and also improve your data storytelling skills.
However, most students and beginners get stumped on what to include in their portfolio, as their projects are all the same that their classmates, bootcamp associates and seniors have created. So, in this post I am going to tell you what projects you should have in your portfolio kitty, as well as a list of ideas that you can use to construct a collection of projects that will help you stand out on LinkedIn, Github and in the eyes of prospective hiring managers.
In this post though, I am classifying projects based on skill level along with sample ideas for DIY projects that you can attempt on your own.
On that note, if you are already looking for a job, or about to do so, do take a look at my book “DataScience Jobs“, available on Amazon. This book will help you reduce your job search time and quickly start a career in analytics.
Since I prefer R over Python, all the project lists in this post will be coded in R. However, feel free to implement these ideas in Python, too!
a. Entry-level / Rookie Stage
If you are just starting out, and are not very comfortable with even syntax, your main aim is to learn how to code along with DataScience concepts. At this stage, just try to write simple scripts in R that can pull data, clean it up and calculate mean/median and create basic exploratory graphs. Pick up any competition dataset on Kaggle.com and look at the highest voted EDA script. Try to recreate it on your own, read through and understand the hows and whys of the code. One excellent example is the Zillow EDA by Philipp Spachtholz.
This will not only teach you the code syntax, but also how to approach a new dataset and slice/dice it to identify meaningful patterns before any analysis can begin.
Once you are comfortable, you can move on to machine learning algorithms. Rather than Titanic, I actually prefer the Housing Prices Dataset. Initially, run the sample submission to establish a baseline score on the leaderboard. Then apply every algorithm you can look up and see how it works on the dataset. This is the fastest way to understand why some algorithms work on numerical target variables versus categorical versus time series.
Next, look at the kernels with decent leaderboard score and replicate them. If you applied those algorithms but did not get the same result, check why there was a mismatch.
Now pick a new dataset and repeat. I prefer competition datasets since you can easily see how your score moves up or down. Sometimes simple decision trees work better than complex Bayesian logic or Xgboost. Experimenting will help you figure out why.
Sample ideas –
Survey analysis: Pick up a survey dataset like the Stack overflow developer survey and complete a thorough EDA – men vs women, age and salary correlation, cities with highest salary after factoring in currency differences and cost of living. Can your insights also be converted into an eye-catching Infographic? Can you recreate this?
Simple predictions: Apply any algorithms you know on the Google analytics revenue predictor dataset. How do you compare against the baseline sample submission? Against the leaderboard?
Automated reporting: Go for end-to-end reporting. Can you automate a simple report, or create a formatted Excel or pdf chart using only R programming? Sample code here.
b. Senior Analyst/Coder
At this stage simple competitions should be easy for you. You dont need to be in the top 1%, even being in the Top 30-40% is good enough. Although, if you can win a competition even better!
Now you can start looking at non-tabular data like NLP sentiment analysis, image classification, API data pulls and even dataset mashup. This is also the stage when you probably feel comfortable enough to start applying for roles, so building unique projects are key.
For sentiment analysis, nothing beats Twitter data, so get the API keys and start pulling data on a topic of interest. You might be limited by the daily pull limits on the free tier, so check if you need 2 accounts and aggregate data over a couple days or even a week. A starter example is the sentiment analysis I did during the Rio Olympics supporting Team USA.
You should also start dabbling in RShiny and automated reports as these will help you in actual jobs where you need to present idea mockups and standardizing weekly/ daily reports.
Sample ideas –
Twitter Sentiment Analysis: Look at the Twitter sentiments expressed before big IPO launches and see whether the positive or negative feelings correlated with a jump in prices. There are dozens of apps that look at the relation between stock prices and Twitter sentiments, but for this you’d need to be a little more creative since the IPO will not have any historical data to predict the first day dips and peaks.
API/RShiny Project: Develop a RShiny dashboard using Yelp API, showing the most popular restaurants around airports. You can combine a public airport dataset and merge it with filtered data from the Yelp API. A similar example (with code) is included in this Yelp College App dashboard.
Lyrics Clustering: Try doing some text analytics using song lyrics from this dataset with 50,000+ songs. Do artists repeat their lyrics? Are there common themes across all artists? Do male singers use different words versus female solo tracks? Do bands focus on a totally different theme? If you see your favorite band or lead singer, check how their work has evolved over the years.
By now, you should be fairly comfortable with analyzing data from different datasource types (image, text, unstructured), building advanced recommender systems and implementing unsupervised machine learning algorithms. You are now moving from analyze stage to build stage.
You may or may not already have a job by now. If you do, congratulations! Remember to keep learning and coding so you can accelerate your career further.
If you have not, check out my book on how to land a high-paying ($$$) Data Science job job within 90 days.
Look at building Deep learning using keras and apps using artificial intelligence. Even better, can you fully automate your job? No, you wont “downsize” yourself. Instead your employer will happily promote you since you’ve shown them a superb way to improve efficiency and cut costs, and they will love to have you look at other parts of the business where you can repeat the process.
Sample project ideas –
Build an App: College recommender system using public datasets and web scraping in R. (Remember to check terms of service as you do not want to violate any laws!) Goal is to recreate a report like the Top 10 cities to live in, but from a college perspective.
Start thinking about what data you need – college details (names, locations, majors, size, demographics, cost), outlook (Christian/HBCU/minority), student prospects (salary after graduation, time to graduate, diversity, scholarship, student debt ) , admission process (deadlines, average scores, heavy sports leaning) and so on. How will you aggregate this data? Where will you store it? How can you make it interactive and create an app that people might pay for?
Upwork Gigs: Look at Upwork contracts tagged as intermediate or expert, esp. the ones with $500+ budgets. Even if you dont want to bid, just attempt the project on your own. If you fail, you will know you still need to master some more concepts, if you succeed then it will be a superb confidence booster and learning opportunity.
Audio Processing: Use the VOX celebrity dataset to identify the speaker based on audio/speech dataset. Audio files are an interesting datasource with applications in customer recognition (think bank call centers to prevent fraud), parsing for customer complaints, etc.
Build your own package: Think about the functions and code you use most often. Can you build a package around it? The most trending R-packages are listed here. Can you build something better?
Do you have any other interesting ideas? If so, feel free to contact me with your ideas or send me a link with the Github repo.