Commit Graph

93 Commits

Author SHA1 Message Date
a713a110f7 [*] amend comment 2023-05-03 10:16:49 +01:00
2fdf730263 [*] Stats now return nanoseconds timestamps 2023-05-03 08:57:21 +01:00
6683d3cfdc [+] IProtocolPiece.GetNextPiece
[+] IProtocolPiece.ReallocateDrainBuffer
[*] Harden protocol piece use after destroy
2023-04-29 07:00:29 +01:00
d791725438 [*] remove extra whiteline introduced in da69428d 2023-04-28 02:44:39 +01:00
da69428d7e [+] IProtocolStack::PrependInterceptorEx
[+] IProtocolStack::PrependSingleFrameProcessor
[+] IProtocolStack::PrependSingleFrameProcessorEx
[+] IProtocolStack::GetArrayOfInterceptors
[+] IProtocolStack::GetInterceptorAtIndex
[+] IProtocolPiece::ToInputWriter
[+] IProtocolPiece::GetExtendedInterceptor
[+] IProtocolPiece::GetShortPipeInterceptor
2023-04-28 02:41:45 +01:00
b8540c77a7 [*] improved network error reporting 2023-04-26 20:13:17 +01:00
cdf5a499cb [*] Improved pipe work awareness under protocol stack to allow for propagation of error conditions such that the top most interceptor can kill the pipe work 2023-04-25 12:13:16 +01:00
664eb270dc [+] IIOPipeWork::GetBytesProcessedInterframe
[*] IOPipeProcessors will now cap reported progression to the requested length if the requested page size would mean we overrun it.
2023-04-25 07:38:36 +01:00
4b0dfabd67 [*] Comments 2023-04-21 05:58:24 +01:00
b2311a8824 [+] IProcessSectionMapView::LockSwap
[+] IProcessSectionMapView::UnlockSwap
[*] Fix critical tag under ILogger
[*] Added missing includes to experimental APIs
2023-04-17 15:55:51 +01:00
f43251c8fc [+] AuNet::ISocketChannelEventListener
[+] AuFS::UpdateTimes
[+] AuFS::UpdateFileTimes
[+] AuFS::CompressEx
[*] AuFS::Compress now rejects files that look to be already compressed
[+] AuFS::DecompressEx
[+] AuFS::Create
[+] AuFS::WriteNewFile
[+] AuFS::WriteNewString
[+] AuFs::FileAttrsList
[+] AuFs::FileAttrsGet
[+] AuFs::FileAttrsSet
[+] DirectoryLogger::uMaxLogsOrZeroBeforeCompress
[+] ISocketChannel.AddEventListener
[+] ISocketChannel.AddEventListener
[+] DirectoryLogger.uMaxLogsOrZeroBeforeCompress
[*] Fix UNIX regression
[*] Fix up stream socket channel realloc IPC
[*] Fix shutdown regression in pretty much everything thanks to 8ff81df1's dumbass fix
    (fixes fence regression on shutdown)
[*] Fix DirDeleterEx formatting of reported failed paths
[*] Fix up file not truncated if already exists bugs. Extended and alternative apis added.
[*] Fix ICompressionStream::ReadEx returning the wrong read value
[+] Legacy compression API can now self-correct once newer stream processor objects are added
2023-02-04 19:43:01 +00:00
04aca5fcf2 [+] Aurora::IO::Net::NetSocketConnectByHost
[+] Aurora::IO::FS::DirDeleterEx
[+] Aurora::IO::Compress
[+] Aurora::IO::Decompress
[*] Aurora::Memory::ByteBuffer zero-alloc fixes
[*] Aurora::Memory::ByteBuffer linear read of begin/end should return (`const AuUInt8 *`)'s
[*] Changed NT file CREATE flags
[*] Fix linux regression
[*] Update logger sink DirLogArchive
... [+] DirectoryLogger::uMaxLogsOrZeroBeforeDelete
... [+] DirectoryLogger::uMaxCumulativeFileSizeInMiBOrZeroBeforeDelete
... [+] DirectoryLogger::uMaxCumulativeFileSizeInMiBOrZeroBeforeCompress
... [+] DirectoryLogger::uMaxFileTimeInDeltaMSOrZeroBeforeCompress
... [+] DirectoryLogger::uMaxFileTimeInDeltaMSOrZeroBeforeDelete
[*] FIX: BufferedLineReader was taking the wrong end head
(prep) LZMACompressor
[*] Updated build-script for LZMA (when i can be bothered to impl it)
(prep) FSOverlappedUtilities
(prep) FSDefaultOverlappedWorkerThread | default worker pool / apc dispatcher / auasync dispatcher concept for higher level overlapped ops
(stub) [+] Aurora::IO::FS::OverlappedForceDelegatedIO
(stub) [+] Aurora::IO::FS::OverlappedCompress
(stub) [+] Aurora::IO::FS::OverlappedDecompress
(stub) [+] Aurora::IO::FS::OverlappedWrite
(stub) [+] Aurora::IO::FS::OverlappedRead
(stub) [+] Aurora::IO::FS::OverlappedStat
(stub) [+] Aurora::IO::FS::OverlappedCopy
(stub) [+] Aurora::IO::FS::OverlappedRelink
(stub) [+] Aurora::IO::FS::OverlappedTrustFile
(stub) [+] Aurora::IO::FS::OverlappedBlockFile
(stub) [+] Aurora::IO::FS::OverlappedUnblockFile
(stub) [+] Aurora::IO::FS::OverlappedDelete
2023-01-26 21:43:19 +00:00
bf8c1eb8c7 [+] AuFS::TrustFile
[+] AuFS::UnblockFile
[+] AuFS::TrustFile
[*] Fixup UNIX open logic a bit

(amended. i did not mess up octals >:( )
2023-01-25 16:50:53 +00:00
1e4082c02f [+] Linux datagrams
[+] NetSocketBind::uDefaultInputStreamSize
[+] NetDatagramBind::uDefaultInputStreamSize
2022-12-16 17:18:12 +00:00
766be57a46 [+] Linux build [again]
[+] ProcessSectionViewReserved.Unix.cpp
[*] Fix missing ::Flush() member on ViewWriter
2022-12-16 00:41:01 +00:00
85a575eb95 [+] Network interfaces API
[+] Logger IFormatter API
[*] Clean up
2022-12-14 20:08:52 +00:00
267c2216b0 [+] UDP over socket API via existing INetSrvDatagram layer
(...missing send)
[+] AuIO::Buffer::ViewReader
[+] AuIO::Buffer::ViewSeekableReadable
[+] AuIO::Buffer::ViewWriter
[*] Clean up AuCompression
[*[ AuLog messages must always crunch for memory
[*] Various bug fixes
[*] Refactor+clean up
2022-12-12 23:50:05 +00:00
1ff9feb303 [+] Initial heap stat counter API
[+] Upcoming failure categories
[+] Updated SysPushXXX prototypes
[*] Restore bandwidth OnTick position for extrapolate (using current frame stats, ref to last) fractional lerps into the future with ref to average
[*] Compression.cpp AuList<AuUInt8> upgrade was incomplete & could've been improved with modern apis
2022-12-08 19:34:15 +00:00
7be2d3fbdc [+] AuUtility::ThroughputCalculator
[+] AuNet::ISocketStats
[+] AuNet::ISocketChannel::GetRecvStats()
[+] AuNet::ISocketChannel::GetSendStats()
[+] AuIO::IOProcessor::RunTickEx(AuUInt32 dwTimeout)
[*] Refactor clock APIs
[+] Documentation in headers
[+] AuIO::IIOPipeWork::GetStartTickMS()
[+] AuIO::IIOPipeWork::GetLastTickMS()
[+] AuIO::IIOPipeWork::GetPredictedThroughput()
[+] AuIO::IIOPipeWork::GetBytesProcessed()
2022-12-06 22:58:15 +00:00
72dc0d715e [*] Begin enforcing steady time
[+] IOProcessor::WakeupThread
[+] NT: Begin hacking in timeBeginPeriod (must spam it in some places)
[+] ConsoleTTY (more specifically the win32 calls) are too slow to run on the mainthread. Delegate to worker.
[*] AuTime.CurrentClockSteady
[*] AuTime.CurrentClockSteadyMS
[*] AuTime.CurrentClockSteadyNS
2022-11-28 16:01:08 +00:00
bb9c383aee [+] ICompressionInterceptor::LimitPassthroughOnOverflow
[+] IOPipeRequest::uMinBytesToRead
[+] (secret api intended for unix users) AuIO::NewLSOSHandleEx
[*] Fix quirks when running Gtk under an io processor
...CtxYield shouldn't spin while work (improper breakout on remote update)
[*] EFileOpenMode::eWrite should assume O_CREAT semantics making eReadWrite somewhat redundant. OpenWrite + eWrite should reasonably work with file-appends. it should not mean force create + cucked GetLength().
2022-11-20 10:31:13 +00:00
d9dd1182b9 [+] TLS pinning
[*] ISocket::Shutdown(*bool bNow*), allowing for flush of the send channel when false
[*] Fix StartRead and StartWrite after shutdown (NT)
[*] Amended dead-lock
2022-11-18 04:44:47 +00:00
f86665fd36 [+] Net: TCP servers can now be multi-threaded
[+] Net: Added missing UDP send datagram
[*] IO bug fixes
2022-11-17 20:58:48 +00:00
898c0ced37 [*] Refactoring in progress... 2022-11-17 08:03:20 +00:00
673872f632 [*] Fix critical bug in rwlock (few commits ago. still havent updated submodules so its ok)
[*] IAsyncTransaction::uStartOffset should be u64
2022-11-09 00:33:14 +00:00
e145352920 [+] IProtocolStack::AppendSingleFrameProcessor
[+] IProtocolStack::AppendSingleFrameProcessorEx
[+] ICompressionInterceptor::FlushNextFrame
[+] ICompressionInterceptor::ConfigureAutoFlushPerFrame
2022-11-08 20:04:12 +00:00
07fd9f19f8 [+] NewProtocolStackFromPipe([...], bool bAutoTick)
[+] IOPipeRequestAIO::uStartOffset
[*] Fix Zstd frames being finicky
[*] Fix compression interceptor invalid weak reference to pipe that
[*] Fix WinFileStream::GetOffset using wrong enumeration
[*] Null ByteBuffer flags on init bc not all operators and constructors account for them
[will prevent bugs in the future]
2022-11-07 13:34:28 +00:00
d63571e4b5 [+] AuCompression::CompressionInterceptor
[+] AuCrypto::CBC::CBCContext
[+] AuFS::ReadDirRecursive
[+] AuFS::DirDeleter
[+] AuCrypto::PBKDF2
[+] AuCrypto::AES::CBCEncrypt
[+] AuCrypto::AES::CBCDecrypt
[+] AuCrypto::TDES::CBCEncrypt
[+] AuCrypto::TDES::CBCDecrypt
[+] Optimize read write locks
[*] Added `ContextFlags = CONTEXT_ALL` to Win32 PlatformWalkCallStack
2022-11-06 13:23:22 +00:00
83ee24014f [+] IProtocolPiece (public interface)
[+] IProtocolPiece::Remove
[+] IProtocolPiece::ToNextWriter
[+] IProtocolPiece::GetNextPieceBuffer
2022-09-29 11:45:08 +01:00
efc02d24f7 [*] Refactor RandomDef: bSecure, ::SetQuickRng()
[*] Default initialize net protocols to zero (tcp)
[*] Warning under AuUInt8 &ByteBuffer::operator[](AuUInt idx)
2022-09-27 17:30:26 +01:00
2a3bd735ac [+] AuProtocol: PrependInterceptor[Ex]/AppendInterceptor[Ex] over now removed AddInterceptor[Ex]
(prepend makes sense for outbound stacks, where you might register the compression/crypto stages first, then return an object to the caller who may wish to prepend the input processors. http might need it.)
2022-09-04 01:18:05 +01:00
af03c5cbf3 [+] IProtocolStack::AddEndInterceptor (optimizes away the output stage buffer, instead of preallocating a massive ring buffer, when the AddInterceptorEx variant is supplied with a uOutputBufferSize of zero)
[*] Typo iServerBadMacLimit
2022-09-02 20:58:56 +01:00
ad4b4aa4ae [*] TLS init for later 2022-08-31 18:46:50 +01:00
d68fc7fc91 [+] ByteBuffer::GetNextLinearRead()
[+] ByteBuffer::GetNextLinearWrite()
[-] ByteBuffer::WriterTryGetWriteHeadFor
[+] ITLSPrivateKeyPair
[+] ITLSContext::GetFatalErrorCodeAsString()
[+] Begin to add certificate chains
[*] Clean up TLS
2022-08-30 22:18:15 +01:00
afa7c8f5f8 [+] TLS::GetDefaultCipherSuites
[+] TLS::GetSupportedCipherSuites
[+] TLS::CipherSuiteFromString
[+] TLS::CipherSuiteToString
[+] internal logic for setting up the ciphersuite array
2022-08-29 19:57:24 +01:00
401cf09962 [*] Cleanup/refactor of AuIO 2022-08-29 16:46:46 +01:00
fa0d46d4c7 [+] Protocol::NewProtocolStackFromPipe 2022-08-29 12:09:53 +01:00
67905a4192 [+] Network + Protocol + TLS - Initial Commit
=============================================================================
Network ]====================================================================
=============================================================================

[+] Added (very) early Aurora::IO::Net implementation
[+] AuNet::EHostnameType
[+] AuNet::EIPProtocol
[+] AuNet::ENetworkError
[+] AuNet::ETransportProtocol
[+] AuNet::INetInterface
[+] AuNet::INetSrvDatagram
[+] AuNet::INetSrvResolve
[+] AuNet::INetSrvSockets
[+] AuNet::INetSrvWorkers
[+] AuNet::INetWorker
[+] AuNet::IPAddress
[+] AuNet::IResolver
[+] AuNet::ISocket
[+] AuNet::IResolver
[+] AuNet::ISocketBase
[+] AuNet::ISocketChannel
[+] AuNet::ISocketDriver
[+] AuNet::ISocketDriverFactory
[+] AuNet::ISocketServer
[+] AuNet::ISocketServerDriver
[+] AuNet::NetEndpoint
[+] AuNet::NetError
[+] AuNet::NetHostname
(+implementation)

=============================================================================
Protocol ]===================================================================
=============================================================================

[+] IProtocolInterceptor
[+] IProtocolInterceptorEx
[+] IProtocolStack
(+implementation)

=============================================================================
TLS ]========================================================================
=============================================================================

[+] ITLSContext
[+] TLSProtocolRecv
[+] TLSProtocolSend
(+implementation)

=============================================================================
IO Bug Fixes ]===============================================================
=============================================================================

[*] IOProcessor::SubmitIOWorkItem should signal the CvEvent, forcing at least once future tick (wont optimize with if in tick & not yet dispatched work items)
[*] Split IOPipeWork in into IOPipeProcessor header
[+] IOPipeWork::GetBuffer (internal reallocation)
[*] Harden against IAsyncTransactions without a loop source
[*] Missing null `if (processor->listener)` in IOProcessor
[*] Solved some soft-lock conditions under Linux's LoopQueue (added deferred commits)
[*] Quick hack: IOProcessor::HasItems() should OR the early can-tick check function.

=============================================================================
Other ]======================================================================
=============================================================================

[+] Linux: LSSignalCatcher
[+] `static void AuResetMember(Aurora::Memory::ByteBuffer &ref)` for AuROXTL
[*] Attempt to enforce a normalization and don't overwrite-readptr-under-istreamwriters policy in ByteBuffer_ReadWrite (circular buffers)
[*] Bad ECC ctors

=============================================================================
Known issues ]===============================================================
=============================================================================

> Linux net is nowhere near done
> UDP socket emulation layer isn't implemented
> Ciphersuite API is a stub
> Private key API is a stub
> ...therefore no TLS servers
> Missing thread safety precautions under net
> Net implementation is still beri early
2022-08-28 20:02:06 +01:00
b6b4ef5dd1 [*] Linux: bDirectIO wasn't enabling O_DIRECT (dumb)
[*] Fixed build errors post refactor
[*] Don't create an epoll for a read-poll, reuse fd
[+] AuByteBuffer:Allocate(AuUInt length, AuUInt alignment)
[*] NT+Linux: Ensure IO buffers are aligned to AuHwInfo::GetPageSize()
2022-08-21 04:38:49 +01:00
1341f805b0 [*] Lite clean up/Refactor 2022-08-20 07:53:48 +01:00
0a66e6741b [+] Add IOSleep, IOYieldFor utility functions
[*] Add/clean up header comments
2022-08-05 11:28:34 +01:00
bfd9e339fc [*] Clean up of include headers 2022-08-02 05:58:00 +01:00
b8bcab1bdc [+] IReadDir
[+] AuIOFS::ReadDir
[*] Fix erroneous return false under fs remove (NT)
2022-07-23 07:41:47 +01:00
8fb7b7e1ee [*] IOPipeProcessor improvements
[*] NT async file: error 38 "Reached the end of the file." should be handled as a zero length packet much like ERROR_BROKEN_PIPE for stream read interface consistency
2022-07-20 15:10:07 +01:00
803f097f84 [*] Typos and comments 2022-06-29 14:56:59 +01:00
70418df761 [+] Added extended IO apis
> StartSimpleLSWatchEx
> StartSimpleIOWatchEx
> StartIOWatchEx
[*] Update README
2022-06-28 16:04:07 +01:00
0c3344fe46 [+] IIOSimpleEventListener
[+] IOProcessor::StartSimpleIOWatch(const AuSPtr<IIOWaitableItem> &object, const AuSPtr<IIOSimpleEventListener> &listener)
[+] IOProcessor::StartSimpleLSWatch(const AuSPtr<Loop::ILoopSource> &source, const AuSPtr<IIOSimpleEventListener> &listener)
[*] IOAdapterAsyncStream should reset the transactions IO state upon reaching end of segment (should this be per tick?) or upon stream error - otherwise, we end up spinning on a stuck event forever
[*] Fix non-linear path under read of the AuByteBuffer
[*] Fix various other nonlinear conditions under AuByteBuffer
[*] IOProcessor releases registered io item from queue upon request
[*] Fix ConsoleMessage::Write -> enumeration of color should be casted to a uint8
[+] Error telemetry under async task creation
[*] Fix various lock ups and non-blocking spins related to erroneous InternalRunOne impl. Residual preemptive batching was fucking with modern io.
[*] Cleanup TaskFrom/JobFrom. More work required to clean up legacy piss
2022-06-22 14:42:17 +01:00
7fde7d04fb [+] EPipeCallbackType
[+] IIOBufferedStreamAvailable callback
[+] IIOProcessor singleshot work items / IIOProcessorWorkUnit
[+] IOPipeCallback description of a pipes destination
[+] IOPipeInputData description of a pipes source
[+] IOPipeRequest, IOPipeRequestAIO, IOPipeRequestBasic
[+] IPipeBackend hooks for on start/end hooks of IOPipeRequestBasics
[*] Update IOAdapaterAsyncStream implementation to better support caller buffering
[*] Updated IAsyncStreamReader to include a warm/dequeue API for direct async usage
[*] Fix NT IO regressions
[*] Fix ThreadPool shutdown on an unregistered thread
[*] Fix race condition in Async.NT.cpp & fix signalable state to closely match Linux (dunno how this was passing before)
[*] Refactor IOProcessorWorkUnit -> IIOProcessorWorkUnit
[*] Update experimental header to include the changes
2022-06-21 05:49:36 +01:00
17b1a738ca [*] Solve a few compiler warnings. Mostly unrealistic u32 <-> u64 casts in element iteration and string operations we can reasonably ignore 2022-06-14 17:12:21 +01:00
f166849e9f [TODO: BLOCK 2 PROGRESS] 2022-06-14 02:14:51 +01:00