How "Vigil" Was Built: A Unique Way of Hooking

Most Windows hooks begin by changing the thing that executes. Patch the first bytes of a function. Replace an import. Set a breakpoint. Redirect a call. The code moves somewhere else, and the hook gets control.

Vigil starts from the opposite end. The instruction stays where it is. The imported function can stay where it is. Instead, Vigil changes the process structure that the instruction is about to touch, protects that structure with PAGE_GUARD, and lets the resulting exception identify the access.

No prologue patch. No guessed instruction length. No trampoline in the path being observed. A normal PEB read becomes the hook trigger.

That is the part worth looking at. Vigil is a Windows x64 inspection toolkit built around a guarded fake PEB and a second, more conventional import mediation layer. One side observes how a selected thread reads process state. The other blocks selected outbound Winsock calls and records metadata for selected cryptography calls. Together they show two very different answers to the same question: where can control be intercepted without pretending the entire process is contained?

The Hook Is the Data

A classic inline hook owns execution because the CPU reaches modified code. Vigil's PEB monitor owns an access because the CPU reaches protected data. The distinction sounds small. It changes almost everything about the implementation.

Windows x64 keeps the current thread's Thread Environment Block in GS. At offset 0x60, the TEB contains ProcessEnvironmentBlock, a pointer to the process PEB. Code such as IsDebuggerPresent can eventually read PEB.BeingDebugged. Other code reads NtGlobalFlag, heap pointers, loader state, version fields, or the API set map.

Vigil does not chase every instruction that might read those fields. It changes TEB.ProcessEnvironmentBlock for one thread. The pointer no longer leads to the real PEB. It leads to a private 4 KiB copy owned by Vigil.

PTEB Teb = NtCurrentTeb();
PVOID RealPeb = Teb->ProcessEnvironmentBlock;
PVOID FakePeb = VirtualAlloc(
    nullptr,
    0x1000,
    MEM_COMMIT | MEM_RESERVE,
    PAGE_READWRITE
);

CopyMemory(FakePeb, RealPeb, 0x1000);
Teb->ProcessEnvironmentBlock = static_cast<PPEB>(FakePeb);

The actual code is stricter than this reduced example. It reads memory through an internal checked path, owns a module reference while active, validates lifecycle state, allocates a TLS index, and permits only one attached thread. But the pivot is this simple. Change one pointer in one TEB, and ordinary PEB consumers on that thread walk into Vigil's page.

What Goes Into the Fake PEB

VigilStart copies the first 4 KiB of the real PEB. That preserves the pointers and fields normal code expects. A blank structure would fail almost immediately because the PEB is not just a debugger flag. It leads into heaps, loader lists, process parameters, API set data, and other state used by normal Windows code.

Vigil changes two fields in the copy. The byte at offset 0x002, BeingDebugged, becomes zero. The 32-bit field at offset 0x0BC, NtGlobalFlag, also becomes zero. It separately refreshes CrossProcessFlags at offset 0x050 from the real PEB after its exception handlers are registered.

Offset	Field	Vigil State	Reason
0x002	BeingDebugged	Cleared	Attached code sees a clean debugger byte
0x050	CrossProcessFlags	Copied from the real PEB	The fake page tracks the current process flag state
0x0BC	NtGlobalFlag	Cleared	Common heap debugging bits disappear from this view
Everything else in the first page	PEB state	Copied	Normal consumers still receive a structurally useful PEB

This is a fake view, not a fake process. If the real PEB changes later, the copied page does not become a live mirror by itself. Pointers copied into it can still reference real process objects. Direct process queries still exist. Another thread can read the real page. The useful property is narrower: code following the selected thread's TEB sees a controlled PEB page, and every access to that page can be observed.

Turning a Page Into a Hook

After the thread is attached, Vigil applies PAGE_READWRITE | PAGE_GUARD to the fake PEB. The guard modifier asks the memory manager to raise STATUS_GUARD_PAGE_VIOLATION on the next access. Windows then clears the guard condition so the faulting instruction can succeed when execution continues.

That last detail is the whole problem. A guard page is one-shot. Without a rearm path, Vigil would see the first PEB access and miss every access after it.

The first vectored exception handler checks a strict set of conditions. Vigil must be in the attached state. The exception must come from the owning thread. That thread's TEB must still point to the fake PEB. No previous rearm can be pending. The exception must be a guard-page violation with a fault address inside the 4 KiB fake page.

If those checks pass, Vigil records the event:

• The current thread ID
• The offset inside the fake PEB
• Read, write, execute, or unknown access type
• The instruction pointer from CONTEXT.Rip
• The exact fault address
• A monotonic sequence number

The instruction pointer matters more than the field value. An access to fake PEB + 0x002 says that something checked BeingDebugged. The RIP says which instruction did it. That is the difference between knowing a field was touched and having a useful lead for analysis.

The Two-Handler Rearm

Rearming PAGE_GUARD inside the guard exception handler does not work cleanly. The faulting instruction still needs to retry and access the page. If the guard is restored too early, the same instruction faults again. Infinite exception loops are technically very thorough instrumentation. They are less useful for running the target.

Vigil solves this with the processor trap flag and two vectored handlers. It registers a vectored exception handler with AddVectoredExceptionHandler and a vectored continue handler with AddVectoredContinueHandler.

The sequence looks like this:

1. The selected thread touches the guarded fake PEB.
2. Windows raises STATUS_GUARD_PAGE_VIOLATION and clears the page's guard state.
3. Vigil's exception handler records the access and sets EFlags.TF.
4. The faulting instruction retries and completes against the fake PEB.
5. The CPU raises STATUS_SINGLE_STEP after that instruction.
6. Vigil recognizes its pending rearm, temporarily points the TEB back to the real PEB, and moves rearming into the continue path.
7. The continue handler clears the trap flag, reapplies PAGE_GUARD, and points the TEB back to the fake PEB.
8. Normal execution resumes with the next PEB access armed.

if (Code == STATUS_GUARD_PAGE_VIOLATION && OwnsFault(Address)) {
    RecordPebAccess(Address, Context->Rip);
    ThreadContext.RearmPending = TRUE;
    Context->EFlags |= 0x100;
    return EXCEPTION_CONTINUE_EXECUTION;
}

if (Code == STATUS_SINGLE_STEP && ThreadContext.RearmPending) {
    ThreadContext.RearmPending = FALSE;
    Teb->ProcessEnvironmentBlock = RealPeb;
    ThreadContext.ContinueRearmPending = TRUE;
    return EXCEPTION_CONTINUE_EXECUTION;
}

The real implementation carries one more important bit: TrapFlagIntroduced. Vigil remembers whether it added TF or found it already set. If the trap flag was already active, another debugger or instrumentation path may own the single-step state. Vigil does not guess. It restores the real PEB for that thread, records the failure, and allows normal exception search to continue.

That is a good failure mode. The monitor loses coverage, but the thread is not left pointing at a page Vigil can no longer manage.

Why the Ownership Rules Are Strict

The thread that calls VigilEnterCurrentThread owns the attachment. The same thread must call VigilLeaveCurrentThread. A different thread receives VigilStatusWrongThread. VigilStop also refuses to tear down while a thread remains attached.

This is not API fussiness. The changed pointer lives in one TEB. The TLS entry and rearm state describe that same thread. Letting another thread perform cleanup would make it easy to restore the wrong TEB while the attached thread still executes against guarded memory.

Vigil keeps one global attached context and one attached thread ID. That deliberately limits the system to a single PEB-monitored thread. Supporting many threads would need per-thread fake views or shared-page coordination, per-thread trap state, teardown barriers, and rules for simultaneous guard faults. The current design refuses that complexity instead of hiding it behind a map and hoping exception timing stays friendly.

The result is understandable. One owner. One fake page. One rearm state machine.

Exception Coexistence

Vectored handlers are ordered, but they are not exclusive. Vigil asks to place its handlers first when it registers them. A later component can also ask for first position. A foreign handler can run before Vigil and consume an exception. If it does, Vigil never sees that event.

Vigil records exceptions it observes but does not own, then returns EXCEPTION_CONTINUE_SEARCH. Existing VEH and SEH handling can continue. During stop, Vigil removes only its own VEH and VCH registrations. It does not rebuild or reorder the rest of the process handler chain.

This matters because exception hooks often become process-wide land grabs. Vigil takes the narrower path. It claims only a guard fault inside its fake page and the single-step state tied to its own pending rearm. Everything else keeps moving.

The Other Half of Vigil

The fake PEB monitor is the unusual hook. VigilGuard.dll uses a more familiar mechanism for a different job. It patches Import Address Table slots across loaded modules.

Guard has two policies. Selected outbound Winsock calls are blocked with WSAEACCES. Selected CNG, CryptoAPI, DPAPI, and WinTrust calls are allowed to reach the original function, then a metadata event is recorded.

Surface	Hook Action	Recorded Data
Connection, send, and lookup functions	Return access denied	Function, thread, size where available, status, timestamp, sequence
BCrypt and NCrypt	Call original	Broad algorithm class, input size, output size, native status
CryptoAPI and DPAPI	Call original	Function-specific numeric metadata and result
WinTrust	Call original	Function identity and trust result

It does not store plaintext, ciphertext, keys, signatures, hostnames, or address bytes in an event. The algorithm label is inferred from the function family. A BCryptHashData call is categorized as hashing, but Guard does not inspect the algorithm handle and claim it knows which hash was used.

That restraint is useful. Metadata is enough to answer questions such as which thread performed encryption, how large the input was, whether signature verification succeeded, or whether a send was attempted. Copying sensitive buffers into a global telemetry ring would create a different security problem.

How the IAT Layer Finds Calls

At startup, Guard resolves each target export from its provider DLL. The definition table maps a provider and name to a replacement wrapper and storage for the original pointer. It includes kernel32.dll loader functions, Winsock functions, Microsoft socket extensions, CNG, legacy crypto, DPAPI, and WinTrust.

Guard takes a Tool Help module snapshot and walks every loaded image except itself. For each valid AMD64 PE, it parses both the normal import directory and the delay import directory. Matching IAT entries are made writable, replaced atomically with InterlockedCompareExchangePointer, and returned to their previous protection.

Each changed slot is remembered as a tuple:

struct VigilGuardSlot {
    PVOID* Address;
    PVOID Original;
    PVOID Replacement;
};

That tuple is also the cleanup contract. Guard does not blindly write the original pointer back during stop. It uses a compare-exchange that restores the slot only if the slot still contains Guard's replacement. If another component hooked the same import after Guard, that later hook survives.

This is boring ownership, which is exactly what hook teardown needs.

Imports Are Not Enough

A one-time IAT scan covers modules already present. Real programs load DLLs later. They also resolve exports dynamically. Winsock adds another wrinkle because functions such as ConnectEx and WSASendMsg are commonly obtained through WSAIoctl, not a normal import.

Guard covers these paths in layers.

Loader Return Hooks

LoadLibraryA, LoadLibraryW, LoadLibraryExA, and LoadLibraryExW are part of the hook table. Their wrappers call the real loader function, then synchronously inspect and patch the returned module before giving the handle back to the caller.

This closes the common path where a module is loaded and immediately used by the same code. The patch lock serializes the update. An active-load counter prevents stop from restoring global state while one of these wrappers is still working.

Loader Notifications

Guard registers LdrRegisterDllNotification. The callback does not parse and patch the new image while loader state is sensitive. It places the module handle into a fixed queue. A worker thread later takes the entry, resolves any newly available providers, holds a module reference, and patches its imports under the same lock.

The worker also rescans loaded modules roughly every 200 milliseconds. That catches loads which did not pass through a hooked LoadLibrary slot or whose notification path raced with other activity.

Dynamic Export Resolution

GetProcAddress is hooked too. When code asks a known provider module for a covered function by name, Guard returns its wrapper instead of the real export. Ordinal requests pass through to the original function.

This is mediation, not global export patching. A raw function pointer resolved before Guard started remains raw. A custom resolver can walk the export directory itself. Internal provider calls do not need to pass back through another module's IAT. The scope is explicit.

Winsock Extension Pointers

When WSAIoctl receives SIO_GET_EXTENSION_FUNCTION_POINTER, Guard checks the requested GUID. For WSAID_CONNECTEX it returns VigilGuardConnectEx. For WSAID_WSASENDMSG it returns VigilGuardWsaSendMsg. Other control codes continue to the original WSAIoctl.

Without this branch, normal IAT hooking would miss two useful outbound paths. They are discovered as data, so Guard hooks the discovery result.

Events Without an Allocation Storm

Both DLLs use bounded event storage. The fake PEB monitor has a 1,024-entry ring. Guard has a 4,096-entry ring. Events receive monotonic sequence numbers so readers can request activity after a known point.

The Guard writer claims a slot with interlocked operations, publishes the event sequence last, and increments a dropped counter when the slot cannot be claimed. Readers copy an event only when the sequence remains stable before and after the copy. They can inspect DroppedCount and continue from NextSequence.

A bounded ring means old history can disappear. That is honest. An instrumentation DLL should not allocate on every intercepted call and eventually become the largest problem in the process. The consumer gets recent evidence, sequence continuity, and a visible signal when it fell behind.

Getting In Before the Entry Point

VigilLaunch.exe builds the launch sequence around a precise guarantee. It creates the target suspended and under a debugger. It allows the Windows loader and TLS callbacks to finish, plants a temporary breakpoint at the executable entry point, and stops the primary thread there.

Only then does it load Vigil.dll and VigilGuard.dll into the child, resolve their exported APIs, start both runtimes, and attach the primary thread to the fake PEB. After the state is verified, the launcher detaches the debugger and resumes the original entry point.

The order is useful because ordinary application code begins with instrumentation active. It is also a real boundary. Loader work and TLS callbacks already ran. Vigil cannot claim coverage before them. A target that performs important behavior from a TLS callback has a window before either hook layer is ready.

The launcher also does not auto-attach later threads or child processes. Guard's patched import paths apply across threads after their modules are covered. Fake PEB observation remains limited to the primary attached thread.

Failure Is a State

Hook systems become dangerous when partial setup still looks successful. Guard does not continue quietly if a future module cannot be patched. It changes its runtime state to VigilGuardStateFailed, restores the slots it still owns, and releases the module references it held.

The PEB layer follows the same style. Start has rollback. Leave restores the real PEB before clearing thread state. Stop refuses to free the fake page while a thread owns it. If trap-flag coexistence becomes ambiguous or the guard cannot be restored, the attached thread is moved back to the real PEB and monitoring fails open.

None of this makes injected instrumentation harmless. It does make the transitions reviewable. Running, attached, leaving, stopping, failed, and stopped are different states because they have different cleanup obligations.

What Walks Around It

Vigil is not a sandbox. The source says so, and the implementation agrees.

• Only one thread receives the fake PEB pointer.
• The real PEB remains available to process queries and other threads.
• Direct syscalls avoid usermode import hooks.
• Raw pointers resolved before Guard starts remain outside its mediation.
• Manual export parsing can avoid hooked GetProcAddress.
• Custom Winsock providers and internal provider paths may avoid covered imports.
• Manually mapped or otherwise unenumerated code may never be patched.
• Loader notifications and periodic scans leave timing windows.
• Files, registry, local IPC, process creation, devices, and many network mechanisms are not restricted.
• The target can inspect, alter, unload, or unhook in-process instrumentation.

The network layer blocks selected API paths. It does not prove that the process cannot communicate. The crypto layer records selected calls. It does not prove that all cryptography passed through those APIs. A ring event is evidence of a call, not proof of complete data flow.

That distinction keeps the tool useful. A narrow guarantee can be tested. A fake containment claim survives only until the target takes a different path.

It Is Not Stealth

A guarded fake PEB is unusual. It is not invisible. The DLL is loaded. The fake page is allocated. One TEB points somewhere different from the process PEB returned by native queries. The page protection changes. IAT slots point into VigilGuard.dll. Vectored handlers exist. A worker thread scans modules.

VigilAudit.exe treats that observability as something to measure, not something to deny. Its controlled audit compares baseline and injected state and can produce DetectedHighConfidence as a normal verdict. The audit is checking whether module state, fake PEB state, protection, and related artifacts can be found.

That is the right attitude for defensive tooling. If the target shares the process, assume it can see the tool. The goal is controlled inspection in an isolated VM, not a contest over who can unlink the most lists.

Why This Hooking Model Is Useful

The fake PEB technique is useful when the question is not just what value code received. The better question is who asked for it.

A patched IsDebuggerPresent can change one API result. It misses direct GS-relative PEB reads. A hardware breakpoint can watch a few addresses, but debug registers are limited and conspicuous. An inline hook needs a function boundary. A guarded copy turns the entire first PEB page into one observation surface and reports the exact offset and RIP for each access.

The tradeoff is exception cost and thread scope. Every observed access takes a guard exception and a single-step path. Hot PEB traffic becomes expensive. Only the attached thread gets the view. Foreign exception machinery can interfere. Those are not small details, but they are visible details.

Vigil's interesting idea is not that it found a secret Windows hook. It changed the object behind a trusted thread-local pointer, then used page state as the interception boundary. The IAT layer follows the same broader principle in a more conventional form. Control the pointers a program already trusts. Observe the path. Keep the ownership rules strict. Admit every route you do not cover.

The code being watched does not have to move. The view underneath it does. That is the hook.