Data Science – Page 4

Publish A Interactive Data Visualization Dashboard On A Website

Publish An Interactive Data Visualization Dashboard On A Website – All The Solutions I Tried

Data Science, Programming, Python | 2024-09-19

Python emerges as the preferred language for data analytics projects due to its versatility in web crawling and broader applications. Hosting dashboards on personal websites presents traffic management challenges. While Power BI and Tableau have limitations, Grafana Labs offers a viable open-source option. PythonAnywhere provides a tailored solution for Python-based projects, offering affordable plans and recently acquired by Anaconda for enhanced stability and development.

Publish An Interactive Data Visualization Dashboard On A Website – All The Solutions I Tried Read More »

Why Is A Portfolio Website Important For A Data Scientist?

Data Science | 2024-09-19

Building a personal portfolio website is crucial for data scientists and programmers. It showcases side projects, demonstrating expertise and solving unique problems. Owning a website offers opportunities to expand, implement practical features, and learn new tools. My Google Trends Enhanced project exemplifies this approach, providing value while embracing imperfection for timely launch.

Why Is A Portfolio Website Important For A Data Scientist? Read More »

Python Custom CSV Reader Function To Prevent Codec Error

Data Science, Python | 2024-11-02

When you try to read a csv file in Python, pd.read_csv() is the most common method. However, the process of decoding sometimes causes errors as below, and it may not work even you tune several arguments.

Python Custom CSV Reader Function To Prevent Codec Error Read More »

List All Subdirectories And Files In Python, Then Assign Depth And Extensions.

Data Science, Python | 2024-09-19

Iterating a path to list all subdirectories and files is an extremely helpful function. You can read bulk excel files with several pretty simple codes. As a lazy guy like me, wrap it to a function make me write code effectively. The code below seems huge, but it contains multiple pieces of functions. All of them are used to operate lists, and the main function to list paths and files is os_get_dir_list()

List All Subdirectories And Files In Python, Then Assign Depth And Extensions. Read More »

R lubridate ymd Function in Python, Convert Integer To Date

Data Science, Python | 2024-09-19

So many Python programmers complained about the inconvenience of datetime package. This function was inspired by the lubridate library in R, and there is a ymd() that can convert an integer series to date.

R lubridate ymd Function in Python, Convert Integer To Date Read More »

project-management-skills-in-data-science-competitions

Project Management Skills Learned from Shopee Data Analysis Competition

Data Science | 2025-05-31

In Shopee’s data analysis competition, tight time constraints were the biggest challenge. I recommend dividing teams into Data Handlers, Utility Player (PM), and Model Builder, with multiple contingency plans. When internet download speeds are slow, split into WiFi and mobile data groups. Proper process planning and risk management are essential for completing tasks within limited time.

Project Management Skills Learned from Shopee Data Analysis Competition Read More »

Target_Apyori – Improved Apriori Algorithm For Specific Products In Python

Data Science, Python | 2024-09-19

Apriori is a common algorithm for retail analysis. But it will be heavy if you load huge transaction data. You may want to focus on association rules of specific SKU (or called CUG) in some cases. Based on apyori package 1.1.2 for Python, I hacked it to be liter and faster. Most of the arguments are the same with apyori, but with an additional list argument, target.

Target_Apyori – Improved Apriori Algorithm For Specific Products In Python Read More »

download-stock-historical-data-with-python-and-yahoo-finance-api

Retrieve Stock Historical Data With Python And Yahoo Finance API

Data Science, Investment, Python | 2024-09-19

Stock market data is valuable for analysis, but accessing it can be challenging. Yahoo Finance offers a convenient alternative, allowing users to search by stock symbols. While the free API occasionally encounters errors, it remains a viable option for manual retrieval. For automated tasks, try/except statements can help manage connection issues. Other API services exist, but verifying supported symbols is crucial before investing.

Retrieve Stock Historical Data With Python And Yahoo Finance API Read More »

Advanced R Tutorial – Use SDcols In data.table

Data Science, R | 2020-12-05

SDcols in data.table can be tricky to remember. Here are handy examples for quick reference:

1. Use lapply with .SDcols to replace NA values with 0.
2. Aggregate data by summing specific columns using .SDcols.

Keep these snippets for easy copying when needed in your data manipulation tasks.

Advanced R Tutorial – Use SDcols In data.table Read More »

R Tutorial – Source Function Files With Local Attributes

Data Science, Others, R | 2020-12-05

If you are building your own modules or called codebase, it is common to split some huge functions as independent files. In other words, you may source functions in functions. In my case, I have a file named codebase, which includes many short functions. Also, I built some huge modules based on the codebase. However, when I keep improving the codebase, it is hard to update all huge modules to support the latest version of the codebase. So, the following code is really useful. Besides, it’s a great way to prevent conflicts for the same name functions.

R Tutorial – Source Function Files With Local Attributes Read More »