Python
- 10 tips for making the Django Admin more secure
- Django Migrations: A Primer
- How Numba and Cython speed up Python code
Linkdump #69
Linkdump #68
R
- A comparison between spaCy and UDPipe for Natural Language Processing for R users
- Some datasets for teaching data science
- The Postmortem Airtime Effect
- Clustering of Labour MPs based on their voting patterns
Python
- Shallow vs Deep Copying of Python Objects
- faiss – A library for efficient similarity search and clustering of dense vectors
- Distributed Pandas on a Cluster with Dask DataFrames
- A brief tour of Python 3.7 data classes
Interesting articles, projects and news
- Family fun with deepfakes. Or how I got my wife onto the Tonight Show
- Warnung von Ex-Mitarbeitern: Facebook, Google & Co. machen süchtig und gefährden die Gesellschaft
- Rechte und Trump-Anhänger teilen Fake News am häufigsten
- Citizen Science: Das begehrte Wissen der Hobby-Forscher
- A Good Life For All Within Planetary Boundaries (O’Neill, D.W., Fanning, A.L., Lamb, W.F., and Steinberger, J.K) – with interactive data visualizations
- Asking the Right Questions About AI
- “Rechte manipulieren stärker als Linke”: Soziale Medien und die politische Debatte
- “Heatmap” for social athlete’s app reveals secret bases, secret places
- Geschichtsforscher werten Archive mit maschinellem Lernen aus
Vectorization and parallelization in Python with NumPy and Pandas
Modern computers are equipped with processors that allow fast parallel computation at several levels: Vector or array operations, which allow to execute similar operations simultaneously on a bunch of data, and parallel computing, which allows to distribute data chunks on several CPU cores and process them in parallel. When working with large amounts of data, it is important to know how to exploit these features because this can reduce computation time drastically. Taking advantage of this usually requires some extra effort during implementation. With packages like NumPy and Python’s multiprocessing module the additional work is manageable and usually pays off when compared to the enormous waiting time that you may need when doing large-scale calculations inefficiently.
Slides on Topic Modeling – Background, Hyperparameters and common pitfalls
I just uploaded my slides on probabilistic Topic Modeling with LDA that give an overview of the theory, the basic assumptions and prerequisites of LDA and some notes on common pitfalls that often happen when trying out this method for the first time. Furthermore I added a Jupyter Notebook that contains a toy implementation of the Gibbs sampling algorithm for LDA with lots of comments and plots to illustrate each step of the algorithm.
Linkdump #67
R
Python
Interesting articles, projects and news
- Machine Learning: Facebook veröffentlicht ein (weiteres) Tool zur Objekterkennung
- Software used in judicial decisions meets its equal in random amateurs
- Forschung: Künstliche Intelligenz sagt den Todeszeitpunkt voraus
- Die Rattenfänger aus dem Silicon Valley
- Facebook verstärkt den Kampf gegen Hetze und Fake-News
- Medien-Vertrauenskrise: Immer mehr Menschen haben Angst vor Fake News
Linkdump #66
R
- While you wait for that to finish, can I interest you in parallel processing?
- Can’t Stop at 21: Twitter Recipe #22 — Tying Up Loose Threads
- Lime: Local Interpretable Model-Agnostic Explanations (R port of original Python package)
Python
- Lime: Explaining the predictions of any machine learning classifier
- Jupyter notebooks and version control
Interesting articles, projects and news
Linkdump #64
R
- Making R Code Faster : A Case Study
- A curated list of Awesome Shiny Apps for Statistics
- Coding Club R Tutorials
Python
Interesting articles, projects and news
- Indien: Wohl mehr als eine Milliarde Personendaten aus staatlicher Datenbank abgegriffen
- Electricity Map | Live CO2 emissions of electricity consumption
- 34C3: Predictive Policing führt zu Sippenhaft und hebelt die Unschuldsvermutung aus
- 34C3: China – Die maschinenlesbare Bevölkerung
- 34C3: Daten kontra Freiheit
- New York setzt auf diskriminierungsfreie Algorithmen in der Verwaltung
- Künstliche Intelligenz verdrängt menschliche Arbeit in juristischen Berufen
- Künstliche Intelligenz soll Geschehen in Videos verstehen lernen
Linkdump #63
R
- archivist: Boost the reproducibility of your research
- Point Pattern Analysis using Ecological Methods in R
- Animate network data
- Word Vectors with tidy data principles
- Tidy word vectors, take 2!
Python
- Django 2.0 Window expressions tutorial
- Scraping, Geocoding, and Mapping Points with Scrapy, Geopy, and Leaflet
- More than a Million Pro-Repeal Net Neutrality Comments were Likely Faked
- PCA using Python (scikit-learn)
- netgraph — Fork of networkx drawing utilities for publication quality plots of networks
- Requestium — Integration layer between Requests and Selenium for automation of web actions
Interesting articles, projects and news
- Researchers find oddities in high-profile gender studies
- No, you’re not being paranoid. Sites really are watching your every move
- Sensoren, Daten, Algorithmen: Regieren nach Zahlen oder das Ende des Politischen
- Machine Learning: Google bringt TensorFlow Lite für Android und iOS
- Big Data in der Medizin: “Das Smartphone des Patienten ist das Stethoskop des 21. Jahrhunderts”
Web scraping with automated browsers using Selenium
Web scraping, i.e. automated data mining from websites, usually involves fetching a web page’s HTML document, parsing it, extracting the required information, and optionally follow links within this document to other web pages to repeat this process. This approach is sufficient for many websites that display information in a static way, i.e. do not respond to user interaction dynamically by the means of JavaScript. In these cases, web scraping can be implemented with Python packages such as requests and BeautifulSoup. Even interactive elements such as forms can be emulated by observing the HTTP POST and GET data that is send to the server, whenever a form is submitted. However, this approach has limits. Sometimes, it is necessary to automate a whole browser in order to implement web scraping on JavaScript-heavy websites as will be shown with a short example in this post.
Recent Comments