Web - HTTP, HTML, web servers, tools, ...

• I'm trying to setup an Apache virtualhost.
• In /etc/apache2/, my configuration is as basic as it can be, and I know it is correct.
• /etc/hosts has a line such as :

  127.0.0.1	siteName

• If I type / paste, in my browser address bar : http://siteName, I get results such as :
  • nothing :
    • Firefox says :

      Hmm. We're having trouble finding that site.
      
      We can't connect to the server at siteName.
      
      

    • Chromium says :

      This site can't be reached
      
      Check if there is a typo in siteName.
      
      DNS_PROBE_FINISHED_NXDOMAIN

  • OR the browser (both Firefox and Chromium) :
    1. considers http://siteName as a search query
    2. sends it to the configured search engine
    3. display results
    which is NOT what I expected () !

if(some condition ...) {
	var myJavascriptSnippet = document.createElement("script");
	myJavascriptSnippet.setAttribute("type", "text/javascript");
	myJavascriptSnippet.setAttribute("src", "someFile.js");
	document.body.appendChild(myJavascriptSnippet);
	}
else {
	deal with it ...
	}

Install

Configure

https://goaccess.io/


Generate HTML report :
goaccess -a -o foo > /path/to/web/directory/report.html

• HTML pages with linked images (like <img src="path/to/image" />) + efficient caching ?
• or the same HTML pages with base64-embedded images ?

Notes :

The typical answer is : small files should be inlined, large files should be served separately.

The server signature is the short text displaying the web server name, its version, as well as details on the operating system itself on servers using the default HTTP 404 page :

It can also be a dedicated HTTP header : Server: lighttpd/1.4.25, or :

Such signatures disclose valuable information to attackers, which is why it is wise to hide them.

On Apache (source) :

On Lighttpd :

On Varnish (source) :

This custom HTTP header prevents browsers from doing MIME-sniffing. Only Internet Explorer and Chrome/Chromium implement it so far (source).

You may then ask "What exactly is MIME-sniffing ?" :
Content-sniffing aka MIME-sniffing is inspecting the content of a byte stream in order to deduce the file format of the data (source).

mod_pagespeed is an open-source webserver module developed by Google to automatically apply web performance best practices to pages and their assets (CSS, JS, images) without requiring to modify the existing content or workflow. It is only available for Apache and Nginx so far.

Setup (source) :

ModPagespeedModifyCachingHeaders off

Enable mod_pagespeed on a specific virtualhost of an Apache webserver :

About mod_pagespeed's internal cache :

About mod_pagespeed's optimize for bandwidth mode :

Downstream caching :

Serving a maintenance page with a HTTP 503 sounds like a smart idea, but reverse proxies (such as Akamai) can be taught not to cache such pages and serve stale content instead. So a HTTP 302 is the right option.

With Apache

# Netscape HTTP Cookie File
# http://curl.haxx.se/rfc/cookie_spec.html
# This file was generated by libcurl! Edit at your own risk.

admin.web.example.com	FALSE	/	FALSE	0		PHPSESSID		97nvft0hm505i0r9jk3c9sds37
.web.example.com	TRUE	/	FALSE	1385999491	example_com_auth_token	529c9e71d22c25.49344365
.web.example.com	TRUE	/	FALSE	1385999491	is_logged_in		true

• domain : the domain that created and that can read the cookie
• flag : a TRUE|FALSE value indicating if all machines within a given domain can access the cookie. This value is set automatically by the browser, depending on the value you set for domain
• path : the path within the domain that the cookie is valid for
• secure : a TRUE|FALSE value indicating if a secure connection with the domain (HTTPS) is needed to access the cookie
• expiration : the UNIX time that the cookie will expire on. If set to 0, or omitted, the cookie will expire at the end of the session (source)
• name : (explicit)
• value : (explicit)

load a page :

HTTP/1.1 200 OK

reload / refresh a page :

HTTP/1.1 304 Not Modified

HTTP/1.1 200 OK

hard-reload / super-refresh a page :

HTTP/1.1 200 OK

The easy method :

The method "from bare metal" :

Find the Akamai origin server :

X-Cache-Key: /L/1550/14006/15m/www.bmw.com.origin.bmw.com/ cid=__
X-True-Cache-Key: /L/www.bmw.com.origin.bmw.com/ cid=__

X-Cache: TCP_MEM_HIT from a69-31-112-141 (AkamaiGHost/6.2.2-6802992) (-)

• answered by the a69-31-112-141 Akamai server
• TCP_MEM_HIT : retrieved from Akamai's cache, did not pass request to origin
• TCP_MEM_MISS (or TCP_MISS ?) + X-Check-Cacheable == YES : there is a chance you might be the first request to the cacheable content after it reached expiration from Akamai's edge

X-Cache-Key: /=/1924/123456/4h/httpOriginHostName/path/to/resource

• 123456 : CP Code
• 4h : resource TTL within Akamai cache

  This value is read by plugins (such as Exceda Akamai Headers) to display the TTL of any given resource. It may not display the real TTL of the current resource. Indeed, this value is the default cache duration defined in the Akamai property configuration; this value can be overridden by an Expires header, that can also be overridden by a Cache-control header. The real TTL value is available in the HTTP headers. (source)

X-Check-Cacheable: YES

• X-Check-Cacheable == YES : the Akamai configuration makes this resource cacheable (without considering whether it's currently served from the cache)
• X-Check-Cacheable == NO : all requests to this piece of content is being passed directly from Akamai to the origin.

Flags

urlencode text with Bash (source) :

urldecode text with Bash (source) :

http://stackoverflow.com/search?q=urldecode+bash

• check the length of any submitted entry : if the expected data is 10 characters long, anything longer could be dubious. Don't rely on the length="..." attribute of HTML form fields, as an attacker may have built his own form to submit data to your application
• create a white list of characters that are legal to be found in user entries. Use regular expressions to check entries towards this list
• Parse the user input for SQL keywords : SELECT, INSERT, DELETE, UPDATE, FROM, CREATE, GRANT, USE, SHOW, DROP, TRUNCATE, SET, NOT, IN, AND, OR
• check the type of data that is submitted in view of the field containing it : a field expecting an integer or a float should not contain a string
• use as much as possible the database / scripting language built-in functions (such as mysql_real_escape_string for PHP+MySQL) to sanitize the user input
• limit database permissions : whenever possible, create an account per usage type with very restricted rights. For instance, the script checking the user credentials on a login page should have SELECT right only, on the users table only
• if the DB / scripting language allows it, use the prepare statement functions so that parameters passed to the queries are considered "just as data", but are not concatenated with the query itself
• configure error reporting carefully to avoid disclosing sensitive information in error messages