Fixed implementation of the max_restarts limit: It used to be one less than the number of allowed restarts, it now is the number of return(restart) calls per request.
The cli_buffer parameter has been removed
Added back umem storage for Solaris descendants
The new storage backend type (stevedore) default now resolves to either umem (where available) or malloc.
Since varnish 4.1, the thread workspace as configured by workspace_thread was not used as documented, delivery also used the client workspace.
We are now taking delivery IO vectors from the thread workspace, so the parameter documentation is in sync with reality again.
Users who need to minimize memory footprint might consider decreasing workspace_client by workspace_thread.
The new parameter esi_iovs configures the amount of IO vectors used during ESI delivery. It should not be tuned unless advised by a developer.
Support Unix domain sockets for the -a and -b command-line arguments, and for backend declarations. This requires VCL >= 4.1.
return (fetch) is no longer allowed in vcl_hit {}, use return (miss) instread. Note that return (fetch) has been deprecated since 4.0.
Fix behaviour of restarts to how it was originally intended: Restarts now leave all the request properties in place except for req.restarts and req.xid, which need to change by design.
req.storage, req.hash_ignore_busy and req.hash_always_miss are now accessible from all of the client side subs, not just vcl_recv{}
obj.storage is now available in vcl_hit{} and vcl_deliver{}.
Removed beresp.storage_hint for VCL 4.1 (was deprecated since Varnish 5.1)
For VCL 4.0, compatibility is preserved, but the implementation is changed slightly: beresp.storage_hint is now referring to the same internal data structure as beresp.storage.
In particular, it was previously possible to set beresp.storage_hint to an invalid storage name and later retrieve it back. Doing so will now yield the last successfully set stevedore or the undefined (NULL) string.
IP-valued elements of VCL are equivalent to 0.0.0.0:0 when the connection in question was addressed as a UDS. This is implemented with the bogo_ip in vsa.c.
beresp.backend.ip is retired as of VCL 4.1.
workspace overflows in std.log() now trigger a VCL failure
workspace overflows in std.syslog() are ignored
added return(restart) from vcl_recv{}
The alg argument of the shard director .reconfigure() method has been removed - the consistent hashing ring is now always generated using the last 32 bits of a SHA256 hash of "ident%d" as with alg=SHA256 or the default.
We believe that the other algorithms did not yield sufficiently dispersed placement of backends on the consistent hashing ring and thus retire this option without replacement.
Users of .reconfigure(alg=CRC32) or .reconfigure(alg=RS) be advised that when upgrading and removing the alg argument, consistent hashing values for all backends will change once and only once.
The alg argument of the shard director .key() method has been removed - it now always hashes its arguments using SHA256 and returns the last 32 bits for use as a shard key.
Backwards compatibility is provided through vmod blobdigest with the key_blob argument of the shard director .backend() method:
for alg=CRC32, replace:
<dir>.backend(by=KEY, key=<dir>.key(<string>, CRC32))
with:
<dir>.backend(by=BLOB, key_blob=blobdigest.hash(ICRC32, blob.decode(encoded=<string>)))
Note: The vmod blobdigest hash method corresponding to the shard director CRC32 method is called ICRC32
for alg=RS, replace:
<dir>.backend(by=KEY, key=<dir>.key(<string>, RS))with:
<dir>.backend(by=BLOB, key_blob=blobdigest.hash(RS, blob.decode(encoded=<string>)))
The shard director now offers resolution at the time the actual backend connection is made, which is how all other bundled directors work as well: With the resolve=LAZY argument, other shard parameters are saved for later reference and a director object is returned.
This enables layering the shard director below other directors.
The shard director now also supports getting other parameters from a parameter set object: Rather than passing the required parameters with each .backend() call, an object can be associated with a shard director defining the parameters. The association can be changed in vcl_backend_fetch() and individual parameters can be overridden in each .backend() call.
The main use case is to segregate shard parameters from director selection: By associating a parameter object with many directors, the same load balancing decision can easily be applied independent of which set of backends is to be used.
To support parameter overriding, support for positional arguments of the shard director .backend() method had to be removed. In other words, all parameters to the shard director .backend() method now need to be named.
Integers in VCL are now 64 bits wide across all platforms (implemented as int64_t C type) , but due to implementation specifics of the VCL compiler (VCC), integer literals' precision is limited to that of a VCL real (double C type, roughly 53 bits).
In effect, larger integers are not represented accurately (they get rounded) and may even have their sign changed or trigger a C compiler warning / error.
Add VMOD unix.
Add VMOD proxy.
We have now defined three API Stability levels: VRT, PACKAGE, SOURCE.
New API namespace rules, see phk_api_spaces_
Rules for including API headers have been changed: * many headers can now only be included once * some headers require specific include ordering * only cache.h _or_ vrt.h can be included
Signatures of functions in the VLU API for bytestream into text serialization have been changed
vcl.h now contains convenience macros VCL_MET_TASK_B, VCL_MET_TASK_C and VCL_MET_TASK_H for checking ctx->method for backend, client and housekeeping (vcl_init/vcl_fini) task context
vcc files can now contain a $Prefix stanza to define the prefix for vmod function names (which was fixed to vmod before)
vcc files can contain a $Synopsis stanza with one of the values auto or manual, default auto. With auto, a more comprehensive SYNOPSIS is generated in the doc output with an overview of objects, methods, functions and their signatures. With manual, the auto-SYNOPSIS is left out, for VMOD authors who prefer to write their own.
All varnish internal SHA256* symbols have been renamed to VSHA256*
libvarnish now has VNUM_duration() to convert from a VCL duration like 4h or 5s
director health state queries have been merged to VRT_Healthy()
Renamed macros: * __match_proto__() -> v_matchproto_() * __v_printflike() -> v_printflike_() * __state_variable__() -> v_statevariable_() * __unused -> v_unused_ * __attribute__((__noreturn__) -> v_noreturn_
- ENUMs are now fixed pointers per vcl
Added VRT_blob() utility function to create a blob as a copy of some chunk of data on the workspace.
Directors now have their own admin health information and always need to have the (struct director).admin_health initialized to VDI_AH_* (usually VDI_AH_HEALTHY).
This is the first beta release of the upcoming 5.0 release.
The list of changes are numerous and will not be expanded on in detail.
The release notes contain more background information and are highly recommended reading before using any of the new features.
Major items:
Changes since 4.1.8:
Changes since 4.1.7:
Changes since 4.1.7-beta1:
Changes since 4.1.6:
Changes between 4.0 and 4.1 are numerous. Please read the upgrade section in the documentation for a general overview.
New since 4.0.2-rc1:
New since 4.0.1:
New since 4.0.0:
New since 4.0.0-beta1:
New since TP2:
varnishsizes -----------~
Persistent storage is now experimentally supported using the persistent stevedore. It has the same command line arguments as the file stevedore.
obj.* is now called beresp.* in vcl_fetch, and obj.* is now read-only.
The regular expression engine is now PCRE instead of POSIX regular expressions.
req.* is now available in vcl_deliver.
Add saint mode where we can attempt to grace an object if we don't like the backend response for some reason.
Related, add saintmode_threshold which is the threshold for the number of objects to be added to the trouble list before the backend is considered sick.
Add a new hashing method called critbit. This autoscales and should work better on large object workloads than the classic hash. Critbit has been made the default hash algorithm.
When closing connections, we experimented with sending RST to free up load balancers and free up threads more quickly. This caused some problems with NAT routers and so has been reverted for now.
Add thread that checks objects against ban list in order to prevent ban list from growing forever. Note that this needs purges to be written so they don't depend on req.*. Enabled by setting ban_lurker_sleep to a nonzero value.
The shared memory log file format was limited to maximum 64k simultaneous connections. This is now a 32 bit field which removes this limitation.
Remove obj_workspace, this is now sized automatically.
Rename acceptors to waiters
vcl_prefetch has been removed. It was never fully implemented.
Add support for authenticating CLI connections.
Add hash director that chooses which backend to use depending on req.hash.
Add client director that chooses which backend to use depending on the client's IP address. Note that this ignores the X-Forwarded-For header.
varnishd now displays a banner by default when you connect to the CLI.
Increase performance somewhat by moving statistics gathering into a per-worker structure that is regularly flushed to the global stats.
Make sure we store the header and body of object together. This may in some cases improve performance and is needed for persistence.
Remove client-side address accounting. It was never used for anything and presented a performance problem.
Add a timestamp to bans, so you can know how old they are.
Quite a few people got confused over the warning about not being able to lock the shared memory log into RAM, so stop warning about that.
Change the default CLI timeout to 10 seconds.
We previously forced all inserts into the cache to be GET requests. This has been changed to allow POST as well in order to be able to implement purge-on-POST semantics.
The CLI command stats now only lists non-zero values.
The CLI command stats now only lists non-zero values.
Use daemon(3) from libcompat on Darwin.
Remove vcl_discard as it causes too much complexity and never actually worked particularly well.
Remove vcl_timeout as it causes too much complexity and never actually worked particularly well.
Update the documentation so it refers to sess_workspace, not http_workspace.
Document the -i switch to varnishd as well as the server.identity and server.hostname VCL variables.
purge.hash is now deprecated and no longer shown in help listings.
When processing ESI, replace the five mandatory XML entities when we encounter them.
Add string representations of time and relative time.
Add locking for n_vbe_conn to make it stop underflowing.
When ESI-processing content, check for illegal XML character entities.
Varnish can now connect its CLI to a remote instance when starting up, rather than just being connected to.
It is no longer needed to specify the maximum number of HTTP headers to allow from backends. This is now a run-time parameter.
The X-Forwarded-For header is now generated by vcl_recv rather than the C code.
It is now possible to not send all CLI traffic to syslog.
It is now possible to not send all CLI traffic to syslog.
In the case of varnish crashing, it now outputs a identifying string with the OS, OS revision, architecture and storage parameters together with the backtrace.
Use exponential backoff when we run out of file descriptors or sessions.
Allow setting backend timeouts to zero.
Count uptime in the shared memory log.
Try to detect the case of two running varnishes with the same shmlog and storage by writing the master and child process ids to the shmlog and refusing to start if they are still running.
Make sure to use EOF mode when serving ESI content to HTTP/1.0 clients.
Make sure we close the connection if it either sends Connection: close or it is a HTTP/1.0 backend that does not send Connection: keep-alive.
Increase the default session workspace to 64k on 64-bit systems.
Make the epoll waiter use level triggering, not edge triggering as edge triggering caused problems on very busy servers.
Handle unforeseen client disconnections better on Solaris.
Make session lingering apply to new sessions, not just reused sessions.
VCL Manual page --------------~
Red Hat spec file ----------------~
Build system -----------~
Build system -----------~
Build system -----------~
The request workflow has been redesigned to simplify request processing and eliminate code duplication. All codepaths which need to speak HTTP now share a single implementation of the protocol. Some new VCL hooks have been added, though they aren't much use yet. The only real user-visible change should be that Varnish now handles persistent backend connections correctly (see ticket #56).
Support for multiple listen addresses has been added.
An "include" facility has been added to VCL, allowing VCL code to pull in code fragments from multiple files.
Multiple definitions of the same VCL function are now concatenated into one in the order in which they appear in the source. This simplifies the mechanism for falling back to the built-in default for cases which aren't handled in custom code, and facilitates modularization.
The code used to format management command arguments before passing them on to the child process would underestimate the amount of space needed to hold each argument once quotes and special characters were properly escaped, resulting in a buffer overflow. This has been corrected.
The VCL compiler has been overhauled. Several memory leaks have been plugged, and error detection and reporting has been improved throughout. Parts of the compiler have been refactored to simplify future extension of the language.
A bug in the VCL compiler which resulted in incorrect parsing of the decrement (-=) operator has been fixed.
A new -C command-line option has been added which causes varnishd to compile the VCL code (either from a file specified with -f or the built-in default), print the resulting C code and exit.
When processing a backend response using chunked encoding, if a chunk header crosses a read buffer boundary, read additional bytes from the backend connection until the chunk header is complete.
A new ping_interval run-time parameter controls how often the management process checks that the worker process is alive.
A bug which would cause the worker process to dereference a NULL pointer and crash if the backend did not respond has been fixed.
In some cases, such as when they are used by AJAX applications to circumvent Internet Explorer's over-eager disk cache, it may be desirable to cache POST requests. However, the code path responsible for delivering objects from cache would only transmit the response body when replying to a GET request. This has been extended to also apply to POST.
This should be revisited at a later date to allow VCL code to control whether the body is delivered.
Varnish now respects Cache-control: s-maxage, and prefers it to Cache-control: max-age if both are present.
This should be revisited at a later date to allow VCL code to control which headers are used and how they are interpreted.
When loading a new VCL script, the management process will now load the compiled object to verify that it links correctly before instructing the worker process to load it.
A new -P command-line options has been added which causes varnishd to create a PID file.
The sendfile_threshold run-time parameter's default value has been set to infinity after a variety of sendfile()-related bugs were discovered on several platforms.
The formatting callback has been largely rewritten for clarity, robustness and efficiency.
If a request included a Host: header, construct and output an absolute URL. This makes varnishncsa output from servers which handle multiple virtual hosts far more useful.
The flag that is raised upon reception of a SIGHUP has been marked volatile so it will not be optimized away by the compiler.