Hey, friends!
Welcome back to the Deep Dive Diaries Newsletter 👋
You’ll have noticed that last Thursday I didn’t release a newsletter.
The truth is that I was going through some tough things personally.
I wanted to take the time to gather myself, and come back this week with a fresh mindset.
Now I’m ready to go again, and share with you guys all the interesting things I’ve found this week
But first, let’s take a look at what I’ve been up to..
⏰ READ TIME: 3 mins and 59 seconds.
What lies ahead…
What’s New? 🤔
Data Digest: Latest Insights 🚀
Data Science Demystified 🪄
Big Data LDN
A few weeks ago I went to the largest data conference in the UK
Big Data London!
I woke up at 4:15am to get ready and set off
At 5:30am I was on the train and on my way to London
Let me just say..
Getting the tube for the first time was an experience..
But I made it to the venue.
Immediately I was surprised how big the Venue (Olympia) was.
Thousands of people, dozens of speakers, and tonnes of stalls.
Throughout the day I enjoyed talks from many leading professionals.
I connected online with multiple people, and gained a more in-depth understanding of the world of data!
I have to admit something..
.
.
I felt like a complete IMPOSTER.
Imposter syndrome hit me hard!
Most of the information that was presented when straight over my head.
The event was catered for industry professionals with in many cases a decade+ of experience.
However, this didn’t stop me from enjoying every moment, and talking with as many people as possible.
I was reunited with an old friend from my uni days (it’s a small world).
I also visited the Google stand and created an AI image using voice prompts ( see below- I think its beautiful).
.
.
Big Data LDN has given me a broader perspective on the industry, and has only added fuel to my passion for data, and data science.
I will continue to learn all the skills necessary to progress in this field…
I have BIG plans.
.
.
And the best part is that I’m brining you along for the journey!!
Data Digest: Latest Insights 🚀
Machine Learning? Forget it.. there is a new kid on the block
There is now a new approach to financial forecasting called the relevance-based prediction.
This prediction can be applied across sectors including; finance, politics, and sports.
The method emphasises the importance of unusual events or outliers in making predictions.
It does this because they often contain valuable information that common events may not provide.
As Mark Kritzman puts it: “Unusual events contain more information than common events.”
Relevance-based prediction originated from research efforts led by Mark Kritzman at MIT in the late 1990s.
Initially used to measure market turbulence, it has evolved to handle complex relationships in forecasting.
When it comes to making forecasting predictions, many of us would turn to Machine Learning.
ML works very well in many circumstances.
However, when unusual circumstances occur, or something unprecedented happens, it can struggle to take it into account.
As Kirtzman states:
“Model-based machine learning looks at historical data to form a prediction, but if circumstances change in the future — if something unprecedented occurs — then that model is no longer good, and you have to start all over again.”
Some advantages of relevance-based prediction include:
🟠 It Utilises the Mahalanobis distance to quantitatively illustrate the significance of an observation in relation to a prediction, considering both its uniqueness and similarity.
🟠 Provides statisticians with guidance on the level of confidence to assign to a particular prediction.
🟠 Leverages a subset of pertinent observations to generate predictions and offers insights to researchers regarding the optimal sample size required for accuracy.
As data enthusiasts, it’s important to stay up to date with emerging forecasting techniques.
This approach provides more accurate predictions, transparency, quantifies confidence, and helps to optimise performance.
This makes it a valuable tool for us to make sound data-driven decisions and apply them to complex problems.
Data Science Demystified 🪄
The symbiotic relationship of Data Science and Cloud Computing- why they need each other
I found this article on the relationship between Data Science and Cloud computing, and thought with the ever-changing trajectory of data towards the cloud, you’d like to give it a read..
Data Science and Cloud Computing are closely interconnected.
Data Science equips professionals with the skills and knowledge to extract value from data, while Cloud Computing provides the necessary infrastructure for storing and processing this data.
These two fields work together to enable the assessment and execution of projects, driving technological innovation.
The synergy between Data Science and Cloud Computing is expected to strengthen in the future, leading us towards a more data-driven and cloud-powered future.
As a professional working with data, embracing this future is crucial for staying at the forefront of technological advancements.
And that's a wrap for this week, Data Divers!
If you've had a blast diving into this newsletter, why not share it with your friends using this link adamlowther.substack.com
… It's like a secret handshake that lets me know you're loving it and want more!
Also I’d love if you reached out to me on Twitter or by replying to this Email!
Catch you on the flip side, Adam x
My man! Great read! I love hearing about what you're doing...as well as being able to see you out in the trenches making moves in real life. Many stay behind their screen...you are out there! Much love and respect