←

2017 week 2 in programming

A simple demo of phishing by abusing the browser autofill feature

None

The Programmer’s Guide to Booking a Plane

Crawling the Southwest websiteA quick look at Southwest’s site revealed no API.I figured that straight up scraping their website would be an effective approach. It’s printing out fare prices, but I’m not really sure what to do with all of this data. So what did I do? I hooked up Twilio to send me a text message whenever a fare hit a certain threshold. Since I didn’t think it through, I also chose to send the text message at a low interval so that it would keep pinging me until I stopped the script, even if I was out on a date with my lovely wife. Plotting the DataAfter a few days, I got kind of annoyed with looking at a stream of text and thought of complex ways to make it more appealing to look at for the few seconds that I looked at it every day. Booking the FlightDays later, on a normal Tuesday night, it happened: the prices dropped dramatically, something that did not happen every Tuesday, but rather quite randomly. You can check out the project on GitHub if you’d also like to go on a nice vacation.

I Wrote a Faster Sorting Algorithm

These days it’s a pretty bold claim if you say that you invented a sorting algorithm that’s 30% faster than state of the art. I have implemented this sorting algorithm using Timo Bingmann’s Sound of Sorting. So my final sort algorithm actually looks at the number of elements in the collection, and if it’s less than 128 I call std::sort, if it’s less than 1024 I call American Flag Sort, and if it’s more than that I run my new sorting algorithm. So if you’re sorting log messages and you have a lot of messages that start with “Warning:” or “Error:” then my implementation of radix sort would first sort those into separate partitions, and then within each of those partitions it would skip over the common prefix and continue sorting at the first differing character. Is an in-place radix sort algorithm Sorts one byte at a time Falls back to std::sort if a collection contains less than some threshold of items Uses the inner loop of American Flag Sort if a collection contains less than a larger threshold of items Uses Ska Byte Sort if the collection is larger than that Calls itself recursively on each of the 256 partitions using the next byte as the sort key Falls back to std::sort if it recurses too many times Uses std::partition to sort booleans Automatically converts signed integers, floats and char types to the correct unsigned integer type Automatically deals with pairs, tuples, strings, vectors and arrays by sorting one element at a time Skips over common prefixes of collections. The sorting algorithm still succeeded at splitting the data into many smaller partitions, so when I fall back to std::sort, it has a much easier time than it would have had sorting the whole range. Since the interface of this sorting algorithm works differently from existing sorting algorithms, I have to invent new customization points.

Add : asshole driven development

From Agile, to Extreme Programming , to Test Driven Development, the acronyms and frameworks keep piling up. Others say it’s because software people like making things up and can’t help themselves. Asshole-Driven development - Any team where the biggest jerk makes all the big decisions is asshole driven development. Cognitive Dissonance development - In any organization where there are two or more divergent beliefs on how software should be made. Development By Denial - Everybody pretends there is a method for what’s being done, and that things are going ok, when in reality, things are a mess and the process is on the floor. The worse things get, the more people depend on their denial of what’s really happening, or their isolation in their own small part of the project, to survive. Get Me Promoted Methodology - People write code and design things to increase their visibility, satisfy their boss’s whims, and accelerate their path to a raise or the corner office no matter how far outside of stated goals their efforts go.

Grumpy: Go running Python! by Google

None

Network cheatsheet

Source port in 2 bytes, destination port in 2 bytes, sequence number in 4 bytes: in a SYN, this is the client Initial Sequence Number, picked usually randomly, otherwise it is + 1 + number of bytes previously sent, ensuring that packets can be reordered to obtain the original segment. Data offset in 4 bits, the size of the TCP header in 32-bit words, 000, flags in 9 bits: NS, CWR, ECE, URG, ACK, PSH, RST, SYN, FIN, window size in 2 bytes, allowing flow and congestion control, checksum in 2 bytes to check header and data corruption, urgent pointer in 2 bytes pointing to a sequence number, options eg. IPv4 addresses fit in 4 bytes, commonly written in dot-separated decimal, eg. IPv6 addresses fit in 16 bytes, with pairs of bytes represented as colon-separated hexadecimal numbers, with adjacent zeros replaced by :: once in the address. Preamble: 7 bytes to ensure we know this is a frame, not a lower-level header, and to synchronize clocks, Start of Frame Delimiter: 1 byte to break the pattern of the preamble and mark the start of the frame metadata, destination Media Access Control address of the target machine: each machine knows the MAC address of all machines it is directly connected to. Among its 6 bytes, it contains two special bits: the Universal vs. Local bit is 0 if the MAC is separated in 3 bytes identifying the network card’s constructor, and 3 bytes arbitrarily but uniquely assigned by the constructor for each card. EtherType indicates what protocol is used in the payload; if the value is < 1536, it represents the payload size in bytes, payload: up to 1500 bytes of data from the layer above, typically IP, Frame Check Sequence, a common wireless protocol.

Cracking Sublime Text 3

Our goal will be to stop the annoying Sublime Text pop-up from reminding you to buy a license each now and then. If g valid license is 0x0, i.e: FALSE and the pop-up has not been shown for a while, show it. As cmp byte , 0x0 compares g valid license to 0x0 and jne 0x100072b0 jumps to the specified address if the comparison ‘returned’ not equal, i.e: the license is valid, efectively skipping the part of the procedure that shows the pop-up. Aand that’s pretty much it, the license pop-up will not bother you again. Rax = check license(var 30, g license name, g license seats, var 4C); *(int8 t *) g valid license = COND BYTE SET(E);. 0x1: License is valid, as the message Thanks for purchasing will show. 0x2: License is invalid, as the message That license key doesn’t appear to be valid.

Keyboard Controls Key Feature W move up A move left S move down D move right Space pause/resume the snake Esc exit game. Tips: When the snake is running, you could press the Space key to pause the snake and then press the W/A/S/D key to move the snake step by step. Anytime if you want the snake to start running again, just press Space key again. DecideNext(): compute the next move direction D of the snake S1. Compute the shortest path P1 from snake S1’s head to the food. Direct a virtual snake, S2, to eat the food along path P1. Compute the longest path P2 from snake S2’s head to its tail. Compute the longest path P3 from snake S1’s head to its tail. Let D be the direction that moves the snake along the longest path to the food.

Sublime Text vs Visual Studio Code vs Atom Performance Test (Dec 2016)

The reason I sticked with Sublime Text was performance: Atom was slow, even after when Atom 1.0 was announced. These days, I Googled “Sublime Text vs Atom 2016”, trying to see if Atom has significant improvement when I found Visual Studio Code. Visual Studio is also built using Web Technology like Atom, but reviewers said it is faster. Visual Studio Code did not allow me to open “10m lines” file saying “Very large”. Visual Studio Code could not keep syntax highlighting when openning the “1m lines” file. Atom and Visual Studio Code perform significantly worse compared to Sublime Text and TextEdit: launching and Opening windows are seconds slower which it is noticeable and they occupied much more RAM. Visual Studio Code has advantages over Atom in opening files and RAM usage. Sublime Text is still the speedy king, Visual Studio Code second and Atom is the last.

Deep Learning Enables You to Hide Screen when Your Boss is Approaching

Specifically, Keras is used to implement neural network for learning his face, a web camera is used to recognize that he is approaching, and switching the screen. First, let the computer learn the face of the boss with deep learning. Taking face image Recognizing face image Switching screen. Next, we use machine learning so that the computer can recognize the face of the boss. First of all, I need to collect a large number of images for learning. Now, when learned model recognize the face of the boss, I need to change the screen. In this time, I combined the real-time image acquisition from Web camera with face recognition using Keras to recognize my boss and hide the screen.

Cross-Platform Graphics in .NET Core

NET Core and Framework provide the ability to access structs, callbacks, and functions in unmanaged libraries using something called Platform Invoke. Another nice thing about GLFW is the permissive license that allowed me to redistribute the library freely. The first hurdle I knew I had to figure out was to somehow reference different GLFW libraries depending on the platform it was running on as I had a glfw. The name of the library had to be specified at compile time, and could not be decided at runtime. Outputs for each platformAfter searching through many documents, I came across this section of an article about how to include native dependencies in libraries and thought this was just what I wanted. Referencing the right OpenGL librarySo, it turns out that the OpenGL library is not called libopengl32. NET Core, I’d probably take a look at the stuff he is working on.

I’m learning to code at 56. Here’s an epic beat-down of my critical inner self.

What 20-year-old stays with their first developer job that long? Chances are an entry level developer job will pay me less than I’m making now. There’s no guarantee that the job I have now will last. Most programming jobs aren’t even in the technology industry. Me: First things first, I’m going to apply to a lot of jobs, build a network of hiring managers, and make sure I get a lot of interviews. At my current job, I show up every day knowing that somebody at work has something to teach me. That’s how I’ve fit in at every job I’ve had over the last 36 years.

Google’s New Website To Learn Computer Science and Programming

Computers are everywhere in our world today and being an educated citizen requires an understanding of the fundamentals of computer science and its underlying problem-solving methodology of computational thinking. Not every child should become a computer scientist, but all children should have the opportunity to explore and create with computing. Google has developed programs, tools and resources that advance computer science education and provide opportunities for exploration and learning in school, in informal settings and at home.

Alacritty, a GPU-accelerated terminal emulator written in Rust

Alacritty is a blazing fast, GPU accelerated terminal emulator. Alacritty is the result of frustration with existing terminal emulators. Correctness: Alacritty should be able to properly render modern terminal applications like tmux and vim. Performance: Alacritty should be the fastest terminal emulator available anywhere. Alacritty’s biggest claim is that it’s the fastest terminal emulator available. Nominally, Alacritty will draw a new frame whenever the terminal state changes, and only when the state changes. Alacritty is a very fast and usable terminal emulator.

Super Nintendo Entertainment System Features - Pt. 01: Graphics & Palettes

None

How we ported Linux to the PS4

Last year we demoed Linux on the PS4. Since then, we’ve gotten 3D graphics working, polished a number of drivers, and wrote some new ones. Linux on the PS4 is now actually quite a usable OS, with some minor caveats. Continuing last year’s theme of focusing on the Linux port and loader instead of on exploits, this year’s talk focused on the hardware and what it takes to bring up Linux on a very much non-PC-compatible x86 platform. What’s next? Well, last year we said we’d like to upstream some of the work into the Linux kernel, and we still plan to do so. Linux currently boots on it, but the GPU driver needs fixups. As we said last year, we are not planning to release our exploits, as we believe that there are plenty of people capable of developing their own exploits for the PS4 userland and kernel; we are leaving that to the community. We had several people do so throughout the year and that helped fix some Linux bugs and improve ps4-kexec.

TIL - Visual Studio has pascal case based intellisense

None

Visualise your CSS before you write it with cssreference.io

Border-bottom-color border-bottom-left-radius border-bottom-right-radius border-bottom-style border-bottom-width In collection: box-model. Border-bottom border-collapse border-color border-left-color border-left-style border-left-width In collection: box-model. Border-left border-radius border-right-color border-right-style border-right-width In collection: box-model. Border-right border-style border-top-color border-top-left-radius border-top-right-radius border-top-style border-top-width In collection: box-model. List-style-image list-style-position list-style-type list-style margin-bottom In collection: box-model. Outline-color outline-style outline-width outline overflow-wrap overflow-x overflow-y overflow padding-bottom In collection: box-model. Transform-origin transform transition-delay In collection: transitions.

OpenStreetMap plugin for Unreal Engine 4

This plugin allows you to import OpenStreetMap XML data into your Unreal Engine 4 project as a new StreetMap asset type. You can now drag and drop OpenStreetMap XML files into Content Browser to import map data! Of course, there are many other places you can find raw OpenStreetMap XML data on the web also, but keep in mind the plugin has only been tested with files exported directly from OpenStreetMap so far. When you import an OSM file, the plugin will create a new Street Map asset to represent the map data in UE4. All mesh data is generated at load time from the cartographic data in the map asset, including colorized road strips and simple building meshes with triangulated roof polygons. While importing OpenStreetMap XML files, we store all of the data that’s interesting to us in an FOSMFile data structure in memory. Finally, a big thanks to the OpenStreetMap Foundation and the fantastic community who contribute map data and maintain the database.

An absolute beginner’s guide to Machine Learning and Image Classification with Neural Networks

This is a hands-on guide to machine learning for programmers with no background in AI. Using a neural network doesn’t require a PhD, and you don’t need to be the person who makes the next breakthrough in AI in order to use what exists today. Setup and use existing, open source machine learning technologies, specifically Caffe and DIGITS Create a dataset of images Train a neural network from scratch Test our neural network on images it has never seen before Improve our neural network’s accuracy by fine tuning existing neural networks Deploy and use our neural network. This guide won’t teach you how neural networks are designed, cover much theory, or use a single mathematical expression. Installing DIGITS. nVidia’s Deep Learning GPU Training System, or DIGITS, is BSD-licensed Python web app for training neural networks. Fine tuning takes advantage of the layout of deep neural networks, and uses pretrained networks to do the hard work of initial object detection. We’ll throw away the final classification layer(s) and retrain the network with our image dataset, fine tuning it to our image classes. Deciding whether or not it’s worth investing time in learning the theories of machine learning and neural networks is easier when you’ve been able to see it work in a small way.

A Tourist’s Guide to the LLVM Source Code

The core LLVM representation lives in RAM and is manipulated using a large C++ API. This representation can be dumped to readable text and parsed back into memory, but this is only a convenience for debugging: during a normal compilation using LLVM, textual IR is never generated. The second subdirectory, llvm, is a biggie: it contains 878 header files that define all of the LLVM APIs. Projects doesn’t contain anything by default but it’s where you checkout LLVM components such as compiler-rt, OpenMP support, and the LLVM C++ library that live in separate repos. AsmParser: parse textual IR into memory Bitcode: serialize IR into the compact format and read it back into RAM CodeGen: the LLVM target-independent code generator, basically a framework that LLVM backends fit into and also a bunch of library functions that backends can use. DebugInfo is a library for maintaining mappings between LLVM instructions and source code locations. ExecutionEngine: Although LLVM is usually translated into assembly code or machine code, it can be directly executed using an interpreter. Fuzzer: this is libFuzzer, a coverage-guided fuzzer similar to AFL. It doesn’t fuzz LLVM components, but rather uses LLVM functionality in order to perform fuzzing of programs that are compiled using LLVM. IR: sort of a grab-bag of IR-related code, with no other obvious unifying theme.

Getting Past C

One of the medium-term possibilities we’re seriously considering for NTPsec is moving the entire codebase out of C into a language with no buffer overruns, and in general much stronger security and correctness guarantees. These are not going to translate into any language with the correctness properties we want. Another necessary step will be to corral the few remaining platform dependencies into well-isolated library modules, so the vast bulk of the code will in principle translate mechanically into whatever binding of the ANSI/POSIX API the new language supports. There’s lots of web evangelism around both languages, so I’m not going to attempt a detailed comparison here. If it’s not, maybe I can up-gun it until it is - language parsing and translation is a kind of problem I’m pretty good at and enjoy working. In the end it might come down to which language I feel more comfortable in. Despite my huge investment of time in the language, I’m ready.

Imperative vs Declarative Programming

Sadly, you probably encountered a definition similar to this, “You know, imperative programming is like how you do something, and declarative programming is more like what you do, or something.” Let’s go back to the initial definition I made fun of, “Imperative programming is like"How” you do something and declarative programming is more like “What” you do. Before we dive into code, it’s important to realize that many declarative approaches have some sort of imperative abstraction layer. The declarative response to the Red Lobster employee is assuming that the Red Lobster employee knows all the imperative steps to get us to the table. To make the leap more graceful, let’s look at some programming “Languages” that are inherently declarative versus those which are imperative by nature. Declarative Programming is programming with declarations, i.e. declarative sentences. Declarative languages contrast with imperative languages which specify explicit manipulation of the computer’s internal state; or procedural languages which specify an explicit sequence of steps to follow.

TensorKart: self-driving MarioKart with TensorFlow

Project - use TensorFlow to train an agent that can play MarioKart 64. To train a TensorFlow model, you have to define a function for TensorFlow to optimize. Higher level machine learning frameworks might abstract this, but TensorFlow forces the developer to think about what is happening and helps dispel the magic around machine learning. TensorFlow has great documentation and there are plenty of tutorials and source code available. With my model trained, I was ready to let it loose on MarioKart 64. With this in mind I played more MarioKart to record new training data. TensorFlow is super cool despite being a lower level abstraction than tools I used several years ago.

How I Write Tests

I do try to write a lot of tests, these days and I do try to write my code with tests in mind. If your interface is too complex, you can still provide a “Programmable” fake that lets tests configure or plug in its behavior in appropriate ways, instrucing it how to respond to inputs. The “Sarama” Go Kafka client implements such a programmable fake kafka broker, which is used repeatedly in the library’s own tests, and can also be used by consumers wishing to construct more end-to-end tests. In order to make these tests self-contained, and able to express many varieties of tests, GCC developers have built a custom annotation/directive system to instruct the test harness how to interpret this test case, and what to check for. Building a runner for these test cases is a bit of upfront investment, but it’s well worth it; By making it easier to write tests, you’ll ensure that you and other contributors write more tests, and your future selves will thank you for the effort. Regression tests, Regression tests, Regression tests. There are probably nigh-infinitely many bugs you could write, but the set of desired behaviors of your system is relatively finite, and the set of bugs you’re likely to write is similarly limited, and if you turn every bug into a test on one of those behaviors, you’ll eventually converge on testing for most of the important failure modes of your system.