Commit Graph

200 Commits

Author SHA1 Message Date
fa0d46d4c7 [+] Protocol::NewProtocolStackFromPipe 2022-08-29 12:09:53 +01:00
747988e85c [*] Protocol stacks werent allocating ring buffers
[*] Unmerged Linux async changes
2022-08-29 00:39:12 +01:00
67905a4192 [+] Network + Protocol + TLS - Initial Commit
=============================================================================
Network ]====================================================================
=============================================================================

[+] Added (very) early Aurora::IO::Net implementation
[+] AuNet::EHostnameType
[+] AuNet::EIPProtocol
[+] AuNet::ENetworkError
[+] AuNet::ETransportProtocol
[+] AuNet::INetInterface
[+] AuNet::INetSrvDatagram
[+] AuNet::INetSrvResolve
[+] AuNet::INetSrvSockets
[+] AuNet::INetSrvWorkers
[+] AuNet::INetWorker
[+] AuNet::IPAddress
[+] AuNet::IResolver
[+] AuNet::ISocket
[+] AuNet::IResolver
[+] AuNet::ISocketBase
[+] AuNet::ISocketChannel
[+] AuNet::ISocketDriver
[+] AuNet::ISocketDriverFactory
[+] AuNet::ISocketServer
[+] AuNet::ISocketServerDriver
[+] AuNet::NetEndpoint
[+] AuNet::NetError
[+] AuNet::NetHostname
(+implementation)

=============================================================================
Protocol ]===================================================================
=============================================================================

[+] IProtocolInterceptor
[+] IProtocolInterceptorEx
[+] IProtocolStack
(+implementation)

=============================================================================
TLS ]========================================================================
=============================================================================

[+] ITLSContext
[+] TLSProtocolRecv
[+] TLSProtocolSend
(+implementation)

=============================================================================
IO Bug Fixes ]===============================================================
=============================================================================

[*] IOProcessor::SubmitIOWorkItem should signal the CvEvent, forcing at least once future tick (wont optimize with if in tick & not yet dispatched work items)
[*] Split IOPipeWork in into IOPipeProcessor header
[+] IOPipeWork::GetBuffer (internal reallocation)
[*] Harden against IAsyncTransactions without a loop source
[*] Missing null `if (processor->listener)` in IOProcessor
[*] Solved some soft-lock conditions under Linux's LoopQueue (added deferred commits)
[*] Quick hack: IOProcessor::HasItems() should OR the early can-tick check function.

=============================================================================
Other ]======================================================================
=============================================================================

[+] Linux: LSSignalCatcher
[+] `static void AuResetMember(Aurora::Memory::ByteBuffer &ref)` for AuROXTL
[*] Attempt to enforce a normalization and don't overwrite-readptr-under-istreamwriters policy in ByteBuffer_ReadWrite (circular buffers)
[*] Bad ECC ctors

=============================================================================
Known issues ]===============================================================
=============================================================================

> Linux net is nowhere near done
> UDP socket emulation layer isn't implemented
> Ciphersuite API is a stub
> Private key API is a stub
> ...therefore no TLS servers
> Missing thread safety precautions under net
> Net implementation is still beri early
2022-08-28 20:02:06 +01:00
b6b4ef5dd1 [*] Linux: bDirectIO wasn't enabling O_DIRECT (dumb)
[*] Fixed build errors post refactor
[*] Don't create an epoll for a read-poll, reuse fd
[+] AuByteBuffer:Allocate(AuUInt length, AuUInt alignment)
[*] NT+Linux: Ensure IO buffers are aligned to AuHwInfo::GetPageSize()
2022-08-21 04:38:49 +01:00
1341f805b0 [*] Lite clean up/Refactor 2022-08-20 07:53:48 +01:00
42b364840e [*] Cleanup Async.NT.cpp
[*] Fix bug in IOProcessor where work items were not enough to cont execution in a loop checking ::HasItems
[+] PromiseCallbackFunctional constructor with one functor arg
2022-08-18 17:19:32 +01:00
49214dd6f3 [+] Additional keycodes under UNIX ConsoleTTY
[+] Try/catch around LIOS_Process to prevent external exceptions from trashing core io components
2022-08-13 18:32:14 +01:00
f98174b7a2 [*] IOSubmit.Linux.cpp bugfix: save wasted time cancelling triggered polls resulting in a -1 EINVAL 2022-08-10 18:01:02 +01:00
eb144f7377 [+] Quick support for IProcessSectionView on Linux 2022-08-10 12:12:36 +01:00
eda8d4255d [+] IProcessSectionMapView
[+] IProcessSectionView (downgraded bc NT)
2022-08-10 10:50:49 +01:00
64cb7404ba [+] Near 1:1 Linux IPC Pipe compared to the NT equivalent (~= CreateNamedPipeA(nMaxInstances=1, dwOpenMode=PIPE_ACCESS_DUPLEX, dwPipeMode=PIPE_TYPE_BYTE))
[+] Ability to bypass blocking limitation of certain io_submit reads, if the blocking subsystem is a pollable stream (ie: a pipe).
[*] Fixed major Linux bug where LoopQueue items weren't being submitted, if no dequeues were in the same tick
[*] Fix various Linux pipe related bugs
[*] Fix futex bug where the callback was nulled on server-release
2022-08-09 07:48:29 +01:00
d60176afb3 [*] NT IPCPipe: fix up renegotiation after termination 2022-08-09 07:16:09 +01:00
03f072f8d6 [*] BUG: OnPipePartialEvent wasn't sent during IOPipeProcessor ticks (common/all platforms)
[*] Build regression: Win32 clipboard-mode-switcher routines weren't ifdef'd out on non-nt pltforms
2022-08-09 01:55:18 +01:00
13480a0473 [*] NT bug fix: pipe breakages were occurring as a non-block instant readfileex operation, and didn't simulate an async read 0 success.
[*] IOProcessor's stop method now returns true on success instea of true on delegated stop
2022-08-08 16:32:45 +01:00
5dd2be4763 [+] New Linux futex based IPCMutex to better ensure robust breakage 2022-08-07 07:01:55 +01:00
c730d4fc58 [*] Fix: IOPipeProcessor will now report a failure on nominal stream-end, if non-zero request length was not met 2022-08-06 20:56:21 +01:00
0a66e6741b [+] Add IOSleep, IOYieldFor utility functions
[*] Add/clean up header comments
2022-08-05 11:28:34 +01:00
21902a5d5b [+] AuParse::[Stringify/Parse][U/S]Int[16] class of parse APIs
[*] Optimize UNIX IPC ABI: Handle String encodes an array of U16s to optimize space. Could still be better.
2022-08-04 14:08:12 +01:00
e4e0af2f1d [*] Reset stream out variable sooner under filestream 2022-08-02 13:29:01 +01:00
bfd9e339fc [*] Clean up of include headers 2022-08-02 05:58:00 +01:00
fd0c5b51b2 Further Linux support
[+] Begin work on IO futexes for io release on process/thread exit
[+] Linux ::readdir iteration
[+] AuConsole buffering API
[*] Fix sleep as to not get interrupted by signals
[*] Switch the type of FS lock used under Linux
[*] Linux: Use new IPCHandle encoding scheme
[*] Fix undefined behaviour: unintialized timeout values (AuLoop/Linux)
[*] Fix undefined behaviour: ConsoleTTY clear line was called of a color of a random value on stack
[-] Remainings of std dir iterator
[*] Fix pthread_kill (aka send signal to pthread handle) always kills process. This is what you expect bc signal handler inheritance.
[*] Reformat the build Aurora.json file
[+] Added clang warning ignores to the build file
[*] Fix: UNIX need to use STDOUT_FILENO. Was using CRT handle in place of fd by mistake.
[+] Linux implementation for IO yield (AuIO::IOYield() - UNIX::LinuxOverlappedYield())
[*] Fix: Linux async end of stream processing. res 0 = zero bytes consumed. <= was detecting this as an error of code 0. Should succeed with zero bytes.
[+] Linux LoopQueue missing epilogue hook for the IO processor
[*] Various refactors and minor bug fixes
[*] Linux fix: Handle pipe EOS as zero
[*] Linux fix: thread termination via a user signal of 77. Need a force terminate.
[*] IPC handle: fix improper int to bool cast in the header setup within ToString
[*] Linux fix: HWInfo CPU topology regression
[-] Linux fix: remove SIGABRT handler
[*] Missing override in compression, exit, and consoletty headers.
[+] Unix Syslog logger backend
2022-08-02 05:52:57 +01:00
b8bcab1bdc [+] IReadDir
[+] AuIOFS::ReadDir
[*] Fix erroneous return false under fs remove (NT)
2022-07-23 07:41:47 +01:00
da51aaf6a7 [*] sleep deprived 2022-07-21 07:34:55 +01:00
ec4b47c15d [*] Increase IPCHandle checksum bitlength 2022-07-21 07:30:25 +01:00
e24c35d2d0 [*] fix ipc handle checksum rng 2022-07-21 07:25:45 +01:00
febd836d99 [+] Improved IPCHandle implementation 2022-07-21 07:19:45 +01:00
ad4c18abe7 [+] Missing LZ4 compressor
[*] Various compression related bugs under the 1/4th of the AuCompression platform related to compression stream objects. Now all 1/4ths match up.
2022-07-20 21:09:40 +01:00
8fb7b7e1ee [*] IOPipeProcessor improvements
[*] NT async file: error 38 "Reached the end of the file." should be handled as a zero length packet much like ERROR_BROKEN_PIPE for stream read interface consistency
2022-07-20 15:10:07 +01:00
433898709e [*] Check for null handle under LSHandle bc win32 apis are inconsistent as shit
[*] _beginthreadex returns null, nonex returns -1, and it doesnt fucking work with affinity but seems to work with name updates. i dont care to fix this yet. fuck microsoft.
[*] NT: preempt process watch thread of leaders instead of syncing to the process under a watch guard on free (evil, we should use a builtin nt threadpool. also cant be arsed to resolve)
2022-07-08 22:58:49 +01:00
f2791e360b [+] Added end of loop queue poll hook for IO Processor frame tick batching 2022-07-06 04:47:27 +01:00
7fb73ccdb4 [+] IPC Logger: Added optional stream frame header (single u32) option
[*] FIX: Missing `this->isStream = isStream;`
[*] FIX: Blocking APIs under async objects under NT were using illegal calls that were for some reason not failing
2022-07-05 20:48:33 +01:00
97069a1515 [*] Undo fail on pipe broken, restore bytes written = zero behavior for reusable pipes. Will match on Linux. 2022-07-05 04:17:28 +01:00
c0ee48435c [*] More IO bug fixes; a double free, statement that should be uncommented, and wrong IO handle under Async.NT.cpp read 2022-07-04 23:42:02 +01:00
f8c45613a1 [*] I'm a dumbass. This was accounted for a long ass time ago 2022-06-29 21:50:37 +01:00
bc1296a35c [*] Deprecate NT impl of latching completeness of async transactions 2022-06-29 19:47:32 +01:00
803f097f84 [*] Typos and comments 2022-06-29 14:56:59 +01:00
70418df761 [+] Added extended IO apis
> StartSimpleLSWatchEx
> StartSimpleIOWatchEx
> StartIOWatchEx
[*] Update README
2022-06-28 16:04:07 +01:00
250d3fd11e [*] Passive crinkling of today : Async.NT.cpp can now be aborted. Will port identical behavior to Linux. 2022-06-24 20:33:55 +01:00
3a76aa6d69 [*] Fix IOPipeProcessor was checking against base + length instead of write head to determine stream consumer completion
[*] Fixup IPCPipe.NT recycling, including proper disconnect acknowledgment and removal of atomic release hack
[*] End of stream callback hack in Async.NT for IPC. net requires something else; this hack will do
[*] Fix io spin: bShouldReadNext wasn't reset on end of frame
2022-06-22 21:24:04 +01:00
0c3344fe46 [+] IIOSimpleEventListener
[+] IOProcessor::StartSimpleIOWatch(const AuSPtr<IIOWaitableItem> &object, const AuSPtr<IIOSimpleEventListener> &listener)
[+] IOProcessor::StartSimpleLSWatch(const AuSPtr<Loop::ILoopSource> &source, const AuSPtr<IIOSimpleEventListener> &listener)
[*] IOAdapterAsyncStream should reset the transactions IO state upon reaching end of segment (should this be per tick?) or upon stream error - otherwise, we end up spinning on a stuck event forever
[*] Fix non-linear path under read of the AuByteBuffer
[*] Fix various other nonlinear conditions under AuByteBuffer
[*] IOProcessor releases registered io item from queue upon request
[*] Fix ConsoleMessage::Write -> enumeration of color should be casted to a uint8
[+] Error telemetry under async task creation
[*] Fix various lock ups and non-blocking spins related to erroneous InternalRunOne impl. Residual preemptive batching was fucking with modern io.
[*] Cleanup TaskFrom/JobFrom. More work required to clean up legacy piss
2022-06-22 14:42:17 +01:00
7ea75b4014 [*] Until its not 2022-06-21 07:12:11 +01:00
7fde7d04fb [+] EPipeCallbackType
[+] IIOBufferedStreamAvailable callback
[+] IIOProcessor singleshot work items / IIOProcessorWorkUnit
[+] IOPipeCallback description of a pipes destination
[+] IOPipeInputData description of a pipes source
[+] IOPipeRequest, IOPipeRequestAIO, IOPipeRequestBasic
[+] IPipeBackend hooks for on start/end hooks of IOPipeRequestBasics
[*] Update IOAdapaterAsyncStream implementation to better support caller buffering
[*] Updated IAsyncStreamReader to include a warm/dequeue API for direct async usage
[*] Fix NT IO regressions
[*] Fix ThreadPool shutdown on an unregistered thread
[*] Fix race condition in Async.NT.cpp & fix signalable state to closely match Linux (dunno how this was passing before)
[*] Refactor IOProcessorWorkUnit -> IIOProcessorWorkUnit
[*] Update experimental header to include the changes
2022-06-21 05:49:36 +01:00
d47aeda082 [+] EConanonicalInput::eTab
[+] if ^::eString, [+] ...isControlSequence, isAltSequence, isShiftSequence
[*] Update semaphore from class to struct types
[*] GetProfileDomain and frens should check the result string length before returning true
2022-06-16 02:29:55 +01:00
17b1a738ca [*] Solve a few compiler warnings. Mostly unrealistic u32 <-> u64 casts in element iteration and string operations we can reasonably ignore 2022-06-14 17:12:21 +01:00
f166849e9f [TODO: BLOCK 2 PROGRESS] 2022-06-14 02:14:51 +01:00
b029f7692f [*] Linux build errors 2022-06-12 14:53:35 +01:00
5d2b642ac0 [*] Huge refactor - Move Aurora::[IPC and Loop] to Aurora::IO 2022-06-12 01:29:28 +01:00
44108a322e 2/3 of the IO update (very early implementation)
[+] TTYConsole::GetPaddingTopOfLog,GetPaddingHeadOfLog,GetPaddingTopOfLog [+ set variants]
[+] IO::IOYield()
[+] IO::IAsyncTransaction::Failed,GetOSErrorCode()
[+] IByteBufferStreamPair
[+] IIOBufferedInterceptor
[+] IIOBufferedProcessor
[+] IIOEventListener
[+] IIOPipeEventListener
[+] IIOProcessorEventListener
[+] IIOProcessorManualInvoker
[+] IIOWaitableIOLoopSource
[+] IIOWaitableIOTimer
[+] IIOWaitableItem
[+] IIOWaitableTickLimiter
[+] IOAdapterAsyncStream
[+] IOAdapterByteBuffer
[+] IOAdapterCompression
[+] IOAdapterSeeking
[*] Cleanup CpuInfo.Linux.cpp
[*] Fixup async threadpool some more
[*] LSTimer.NT.cpp updates timer object on tick state update, akin to Linux
2022-06-12 00:01:27 +01:00
e29843dc7e [*] Beyond dumb fio offset issue 2022-06-06 22:34:49 +01:00
f182ecd986 [+] Added IOSleep.* files, API: AuIO::WaitFor(ms, waitEntireFrame) 2022-05-13 21:02:19 +01:00
86ac108f11 [*] Move AuIO::WaitMultiple
[+] AuIO::WaitMultiple2
2022-05-13 19:25:29 +01:00
61f20b6c69 [+] Linux: missing new IAsyncTransaction::Reset method 2022-05-05 12:48:37 +01:00
dfc21453ef [+] Add IAsyncTransaction::Reset
[*] Bug fixes: Async.NT.cpp
2022-05-05 11:24:44 +01:00
d5ec986e02 [+] Linux async process stream access API implementation and bug fixes in AuIOFS / Async.Linux.cpp implementation 2022-05-04 20:34:46 +01:00
97cdbc9f27 [+] Overlapped stdout/in/err processes API 2022-05-03 11:10:04 +01:00
237d2d070c [+] Initial attempt at parsing Linux VMA space
[*] Refactor old C++ enums into AUE derivative
2022-04-21 00:34:08 +01:00
f1ef6ac43c [+] Async/overlapped UNIX pipe IO
[*] Further work on Linux LoopQueue for WaitAll
2022-04-17 23:46:05 +01:00
1f15674016 [+] Optional blocking operations for Linux async file objects 2022-04-17 15:51:37 +01:00
371006c04e [+/*] NT IPCPipe, updated interface for bidirectional comms. Introducing linux regression 2022-04-17 13:40:08 +01:00
7a0d9701b5 [*] NT regression and fixes 2022-04-15 19:20:12 +01:00
9f2ff9de19 [+] IPC pipes
[*] Fix socket leak under UNIX fallback path
2022-04-15 15:46:07 +01:00
499aaeedab [+] IPCMutex, IPCSharedMemory
[*] Refactor headers
2022-04-15 11:01:43 +01:00
37cb35d997 [+] UNIX FD sharing (with Linux optimization and a potentially functional BSD fallback)
[+] IPCHandle
[*] Clean up IO init
2022-04-14 20:39:04 +01:00
8468c4d65c [+] Missing UNIX flock advisory impl 2022-04-13 17:20:23 +01:00
415116c891 [+] Linux Async IO 2022-04-13 12:00:35 +01:00
aa7404fc25 [*] Update linux file watcher to emit self modify on dir watches upon file state change much like NT 2022-04-13 09:57:28 +01:00
4617587adb [*] Async file aio API WaitMultiple should return a DWORD and call the callbacks 2022-04-12 20:17:58 +01:00
a36a1b8211 [*] Do not warm dir listings on non-dirs 2022-04-11 13:57:14 +01:00
a3b36ea651 [+] Directory awareness in OPLOCK hack of a watcher to bring NT up to Linux possible quality 2022-04-11 13:54:52 +01:00
2903a9f6cc [*] NT File watcher regression from Linux update 2022-04-11 10:54:51 +01:00
d4e239d27f [+] Did I just implement peeking of NT handles by buffering the signal state? Yes, yes I did.
[*] Further eliminate 32bit MS clock values (we need clock types)
2022-04-11 10:47:09 +01:00
5e82879cc5 [*] Additional NT watcher regression (between API update) 2022-04-11 06:35:13 +01:00
2b798bde97 [*] Win32 build regression (1/2) 2022-04-10 17:20:00 +01:00
92e195e870 [*] Remove debug mask 2022-04-10 16:51:25 +01:00
3defb1bb14 [+] Linux Watcher
[*] Expand watcher API -> Breaking NT
[*] Reexpand loop queue API -> Breaking NT
[*] Linux CPUInfo clean up
[*] Bug fix: mkdir should set execute flag... because directories are special
[*] Refactor: Cleanup base64
[*] Bug fix: UNIX path normalization
[*] Bug fix: missing O_CREAT flag (au auto-creates)
[*] Normalize line endings
2022-04-10 16:40:49 +01:00
9da2147756 [*] Unix signal handlers shouldn't be Linux exclusive 2022-04-07 06:31:38 +01:00
d3428f4cd9 [*] Continued loop work (Linux and Win32) 2022-04-07 02:20:46 +01:00
b97ed198c3 [+] Modern Linux cpuinfo
[*] Replace getpid with gettid
2022-04-06 05:50:12 +01:00
8fe83de42f [*] Linux has a pulse... 2022-04-06 02:24:38 +01:00
48b6994f61 [+] LSFromFdNonblocking
[*] Other Linux progression
2022-04-05 03:36:39 +01:00
5a96a71949 [*] Address *some* build issues under Linux. Interim port progress 2022-04-04 08:53:34 +01:00
62e3490d9f [*] Slight cleanup of the NT watcher 2022-04-03 05:46:52 +01:00
3650599064 [*] Only send ack if requested 2022-04-03 03:44:59 +01:00
603c68f3ca [-] de-bug 2022-04-03 03:26:23 +01:00
a1ec806773 [+] Initial hack of a file watcher in NT land 2022-04-03 02:52:25 +01:00
cfca09d0da [*] FS.cpp bugs regarding expanding "<char>/" 2022-04-03 00:29:49 +01:00
371d01ce71 [*] Missing stage for MakeTemporary 2022-04-03 00:28:18 +01:00
a31136a5d4 [+] MakeTemporary 2022-04-02 19:14:24 +01:00
a4568be6fa [*] Kill net. Branching... 2022-04-01 04:54:21 +01:00
c1976d771b [*] Compiler warning
[*] Disambiguate auContainerUtils
[*] Optimize AuLogXXX / remove alloc
[*] Didn't commit two auROXTL idiom updates (ExceptionWatcher and Logging)
2022-03-30 19:07:15 +01:00
361e8526cf [*] Optimize create-dir-tree for FS benchmark 2022-03-30 15:17:15 +01:00
802361c57e [*] Update FS resources under NT (prevent conflict of privileged user data + brand matching all user data) 2022-03-30 12:33:52 +01:00
ba8602744b [+] Interim and incomplete Unix/Linux buildability 2022-03-22 05:10:40 +00:00
0fb514f856 New memory APIs
[+] AuMemory::Cache
[+] AuMemory::SwapLock
[+] AuMemory::Transition

AuROXTL
[+] AuPageRoundUp
[+] AuPageRound
2022-03-21 05:20:19 +00:00
7bfcce52fd [*] Oops 2022-03-10 16:02:20 +00:00
1f18da1e98 [*] Added missing this-> in StringToProvider for consistency within this file 2022-03-04 23:15:41 +00:00
53e33b6fdd [*] Added/fix UTF-16 BE count
[*] Optimize stage builds - Crypto API validation
[*] Clean up
2022-03-04 22:28:25 +00:00
805eae7f3d [+] Added _AU_SAW_WIN32_EARLY for AuLoop::NewLSHandle in AuWin32Utils.hpp
[*] Clean up AuROXTL
2022-02-27 09:11:56 +00:00
907b5d1b33 [*] Illegal missing return value 2022-02-27 07:46:00 +00:00
9e26996463 [*] Recrinkle win32 process send term and stdin CONIN/OUT win32 logic 2022-02-21 12:27:34 +00:00